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technician  cannot  be  expected  to  master  the  information  base,  after  only  a  few 
months  of  training  and  experience. 

Psychological  approaches  to  the  troubleshooting  problem  attempt  to  improve 
the  selection,  motivation,  and  training  of  technicians.  Among  the  aspects  which 
have  been  considered  are  methods  for  enhancing  the  understanding  of  physical 
relations  In  equipment,  the  hierarchical  analysis  and  practice  of  troubleshoot¬ 
ing  sub-skills,  and  the  general  logic  of  searching  behavior.  The  aiding 
approach  tries  to  "unburden"  the  technician  by  providing  most  of  the  Informa- 
tlon  base  in  special  booklets,  diagrams,  computer  programs,  and  directed  test 
sequences.  The  maintainability  approach  concentrates  on  designing  equipments  so 
that  they  will  be  easy  to  troubleshoot- -for  example,  key  test  points  are  made 
accessible,  and  sub-units  are  arranged  so  as  to  facilitate  diagnosis  and 
replacement.  When  psychological,  aiding,  and  maintainability  technologies  are 
fully  applied,  the  troubleshooting  problem  can  be  "solved",  within  the  present 
state  of  the  art,  for  the  great  majority  of  equipment  items. 

For  troublesome  equipments  which  are  already  In  service,  procedural  1 zed 
aiding  of  troubleshooting  activities  is  probably  the  best  immediate  solution, 
and  many  one-term  military  technicians  might  well  devote  most  of  their  service 
to  application  of  these  aids.  The  aids  must  be  specially  prepared,  tested,  and 
debugged,  and  they  must  be  designed  from  a  behavioral  standpoint.  It  should  be 
recognized,  though,  that  even  the  most  effective  aids  will  not  find  all 
troubles,  and  that  the  services  will  always  need  people  with  relatively  deep 
understanding  of  the  systems  they  have  to  maintain.  To  provide  such  people, 
research  is  needed  on  such  topics  as  the  visualization  and  memory  of  complex 
physical  events,  the  learning  of  long  branching  sequences.  Individual  differen¬ 
ces  in  search  behavior,  and  ways  to  enhance  human  reasoning  by  small  computer¬ 
ized  devices. 
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SUMMARY 


Troubleshooting  of  complex  equipments  in  the  military  remains  a 
problem  for  many  reasons — some  technical,  some  ackninistrative.  At 
the  working  level,  a  major  key  to  success  is  the  technician's 
information  base  or  "cognitive  map"  of  the  relations  that  hold  among 
present  test  symptoms,  normal  indications,  and  the  set  of  possible 
defective  units.  If  this  information  base  is  reasonably  complete  and 
correct,  then  the  technician  can  usually  converge  on  the  trouble  in  a 
reasonable  time.  Many  equipments  are  so  oonplex  that  an  ordinary 
technician  cannot  be  expected  to  master  the  information  base,  after 
only  a  few  months  of  training  and  experience. 

"  Psychological  approaches  to  the  troubleshooting  problem  attempt 
to  improve  the  selection,  motivation,  and  training  of  technicians. 
Among  the  aspects  which  have  been  considered  are  methods  for 
enhancing  the  understanding  of  physical  relations  in  equipment,  the 
hierarchical  analysis  and  practice  of  troubleshooting  sub-skills,  and 
the  general  logic  of  searching  behavior.  The  aiding  approach  tries 
to  "unburden”  the  technician  by  providing  most  of  the  information 
base  in  special  booklets,  diagrams,  computer  programs,  and  directed 
test  sequences.  The  maintainability  approach  concentrates  on 
designing  equipments  so  that  they  will  be  easy  to  troubleshoot — for 
example,  key  test  points  are  made  accessible,  and  sub-units  are 
arranged  so  as  to  facilitate  diagnosis  and  replacement.  When 
psychological,  aiding,  and  maintainability  technologies  are  fully  ^ 


applied,  the  troubleshooting  problem  can  be  "solved,"  within  the 
present  state  of  the  art,  for  the  great  majority  of  equipment  items. 

For  troublesome  equipments  which  are  already  in  service, 
procedural i zed  aiding  of  troubleshooting  activities  is  probably  the 
best  immediate  solution,  and  many  one-term  military  technicians  might 
well  devote  most  of  their  service  to  application  of  these  aids.^jthe 
aids  must  be  specially  prepared,  tested,  and  debugged,  and  they  must 
be  designed  from  a  behavioral  standpoint.  It  should  be  recognized, 
though,  that  even  the  most  effective  aids  will  not  find  all  troubles, 
and  that  the  services  will  always  need  people  with  relatively  deep 
understanding  of  the  systems  they  have  to  maintain.  To  provide  such 
people,  research  is  needed  on  such  topics  as  the  visualization  and 
memory  of  complex  physical  events,  the  learning  of  long  branching 
sequences,  individual  differences  in  search  behavior,  and  ways  to 
enhance  human  reasoning  by  small  computerized  devices. 
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Troubleshooting  Complex  Equipment  in  the  Military  Services: 
Research  and  Prospects 


SECTION  I.  INTRODUCTION 


One  of  the  first  experiments  on  human  troubleshooting  of 
electronic  equipment  was  conducted  in  the  simmer  of  1953  at  the  Long 
Beach  Naval  Shipyard.  Electronics  technicians  from  Navy  ships 
searched  for  faults  inserted  into  radio  and  radar  circuits.  The 
subjects  also  had  to  troubleshoot  a  "simulator"  representation  of  the 
same  circuits.  The  main  idea  was  to  see  if  fault-locating  behavior 
on  the  simulator  correlated  with  performance  on  the  real  equipment; 
in  fact,  it  did  (Grings,  et  al.  1953) . 

That  early  work  was  sponsored  by  the  Office  of  Naval  Research 
because  of  the  dramatic  increase  in  equipment  complexity  that 
occurred  after  world  War  II.  The  new  radars,  fire  control  devices, 
missiles,  analog  computers,  and  communication  sets  were  vastly  more 
conplex,  and  often  more  unreliable,  than  the  equipment  of  the  1940's, 
and  all  the  military  services  found  themselves  with  a  maintenance 
headache.  While  the  increased  capabilities  which  the  advanced 
equipment  afforded  were  highly  desired,  much  of  the  time  the  fancy 
new  gear  was  down,  and  often  there  was  nobody  around  who  knew  how  to 
fix  it.  As  early  as  1953,  Car  hart's  Rand  report  was  calling  the  Air 
Force  maintenance  situation  "intolerable"  (Car hart,  1953) ,  and  one 
could  hear  even  stronger  adjectives  from  operational  military  people 
who  had  to  use  the  new  technology,  and  had  to  live  with  it  when  it 
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failed  to  operate. 

More  than  a  quarter-century  later,  the  troubleshooting  problem  is 

still  with  us.  Ships,  aircraft,  missiles,  and  control  centers  are 

still  plagued  with  unf indable  troubles  and  marginally  performing 

equipment.  Many  expensive  items  never  perform  up  to  design 

expectations,  Or  are  unavailable  at  crucial  moments.  Maintenance 

still  requires  a  lot  of  people  and  a  lot  of  money,  with  nearly  60 

percent  of  total  life-cycle  equipment  costs  attributable  to 

personnel  expenses  (Blanchard,  1979?  Shriver,  1975) .  Maintenance 

difficulties  are  not  confined  to  sophisticated  electronic  systems, 

either.  Consider  this  astounding  quote  from  Secretary  of 

Defense  Schlesinger,  after  the  Mayaguez  incident  of  1975: 

"...The  thirty-one  year  old  carrier  Hancock... operating 
without  one  of  its  four  shafts... never  reached  the  scene. 

The  helicopter  carrier  Okinawa, .. .with  part  of  its  boiler 
plant  off  the  line... also  never  arrived  at  the  scene.  The 
escort  vessel  Holt,  the  first  ship  at  the  scene,  had  power- 
supply  problems,  and  consequently  its  main  battery  was  down 
the  night  before  the  engagement." 

All  of  these  difficulties  occurred  in  a  combat  area  following  a 

major  war,  when  the  Navy  had  days  of  warning.  Evidently,  things  are 

still  "intolerable"  in  military  maintenance. 

This  report  was  undertaken  in  the  belief  that  now  is  a  good  time 

to  take  stock  of  the  troubleshooting  behavior  problem  in  the 

military.  The  whole  field  is  changing  and  particularly  so  in  the 

electronics  domain.  For  one  thing,  we  are  in  an  era  of  sharply 

improved  hardware  reliability.  After  many  years  of  premises  and 

disappointments ,  hardware  is  finally  meeting  its  claims.  These  days, 

a  new  mini-computer  with  dozens  of  terminals  may  run  for  days  or 
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weeks  without  a  main  frame  failure.  This  is  true  "ultra¬ 
reliability."  Already,  integrated-circuit  components  are  manufactured 
to  be  "fault-tolerant,"  so  that  if  something  fails  the  system  keeps 
running  as  before.  In  a  certain  sense,  the  failure  never  happened 
because  the  user  never  knows  it.  But  as  Rouse  (1978)  points  out, 
relatively  rare  breakdowns  mean  that  a  maintenance  technician  will 
not  get  much  practice  in  troubleshooting.  Another  big  change  is  the 
micro-circuit  technology,  which  makes  it  feasible  to  localize  a  fault 
only  down  to  a  board,  can,  or  chip  unit.  At  the  same  time,  cheaper 
and  larger  conputer  memories  make  it  possible  for  system  designers  to 
employ  very  long  operating  and  fault-locating  programs  in  a  small 
unit,  and  this  means  that  a  technician  often  has  to  be  something  of  a 
programmer.  So  we  can  see  that  technicians  will  have  to  become  more 
oriented  to  logic  and  software  and  will  be  less  frequently  concerned 
with  the  old  soldering,  wiring,  and  parts-replacement  skills.  We  are 
probably  at  a  crossroads,  too,  in  the  aiding  and  training  of 
troubleshooters.  Handy  and  comprehensive  "briefcase"  troubleshooting 
guides  are  already  here  (Rigney  &  Towne,  1977) .  A  portable  test  unit 
as  big  as  a  TV  set  can  store  a  whole  book  of  system  information  on  a 
single  floppy  disk  (De  Paul,  1979) ,  and  all  of  the  data  can  be 
interrogated  instantly.  Not  long  from  now,  such  a  compact  guide  will 
tell  the  military  technician  which  check  to  make  next,  will  evaluate 
the  results  of  all  checks  made,  and  will  furnish  a  hard-copy  printout 
of  every  troubleshooting  step.  And  these  aiding  functions  may,  on 
occasion,  be  instantly  transmitted  from  a  renote  ship  or  field  site 
to  an  analysis  center,  for  further  analysis  and  advice.  This  kind  of 
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detailed  aiding  can  be  so  effective  that  much  training  might  well  be 
directed  at  teaching  the  aid,  rather  than  teaching  the  usual  circuit, 
hydraulic,  or  electro-mechanical  theory.  There  will  be  arguments  over 
such  aiding  and  training,  but  it  is  clearly  on  the  military  horizon 
for  the  1980's.  Another  prediction  is  easy  to  make:  more  systems 
will  be  digital.  As  one  writer  asks,  "Is  everything  going  digital? 
Whatever  happened  to  6SN7  tubes,  lew-distortion  amplifiers,  and 
feedback  circuits?"  (Gasperini,  1976) . 

If  indeed  we  are  already  in  an  era  of  ultra-reliable  but  more 
carpi ex  digital  equipment,  in  which  technicians  will  be  aided  in 
their  work  by  extremely  "smart"  test  sets,  then  there  are  many 
interesting  questions.  What  can  we  expect  of  our  troubleshooters? 
Exactly  what  will  their  intellectual  requirements  and  difficulties 
be?  How  much  efficiency  can  we  anticipate?  What  adninistrative  and 
planning  considerations  should  be  most  crucial  regarding  the  military 
technician  force  of  the  next  couple  of  decades?  Hew  should  we  train 
and  "transition"  all  of  the  people  required?  What  contributions  can 
be  made  by  disciplines  such  as  the  psychology  of  reasoning? 

ttiis  report  was  written  primarily  for  the  military  research 
community,  which  is  a  network  of  technical  oaimands,  laboratories, 
contracting  bureaus,  industrial  contractors,  and  academic  institutes. 
As  a  secondary  audience,  there  are  numerous  training,  support,  and 
operational  personnel  who  run  the  technical  schools,  who  have  to  face 
troubleshooting  problems  in  the  field,  or  who  prepare  technical 
Manuals  and  automatic  test  equipment.  Such  people  are  often  engaged 
in  the  delivery  of  a  specific  system,  course,  document  or  procedure. 
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We  also  hope  that  some  military  manpower  planners  will  consider  some 
o£  the  issues  raised  here.  There  are  rather  few  manpower  people  and 
they  are  not  particularly  visible,  but  they  are  the  ones  who  decide 
how  many  Data  Systems  Techs  we  will  have  in  1986,  and  their  decisions 
can  affect  long-range  military  prospects.  There  is  no  need  to  plan 
for  an  expensive  new  ECM  installation  on  a  ship  if  the  prime 
equipments  will  not  be  available  for  use.  With  so  many  big  changes 
in  the  hardware,  software,  and  aiding  capabilities,  manpower 
projections  will  be  more  difficult  and  more  critical,  and  they  should 
be  based  on  what  we  know  about  the  key  behaviors  that  will  be  needed. 

Outline  of  the  Report 

Section  II  examines  the  logic  of  diagnosis,  and  also  reviews 
material  from  academic  psychology  regarding  human-search  performance. 
The  next  chapter  takes  up  aiding  and  training  issues,  many  of  which 
are  being  pursued  right  now  by  DCD  agencies,  laboratories,  and 
contractors.  Section  IV  looks  at  the  maintenance  technician's  work 
from  the  "job  design"  standpoint,  and  explores  his  job  satisfaction 
and  motivation.  The  final  section  lists  seme  research 
reconmendations,  including  nineteen  specific  projects  that  might  be 
considered  by  agencies  like  CNR,  AFOSR,  and  ARI.  There  are  also  a 
few  administrative  reconmendations. 

The  main  conclusion  of  this  report  thesis  can  be  quickly 
imparted.  Troubleshooting  of  very  complex  systems  is  difficult  for 
numerous  reasons,  but  the  critical  factor  is  that  the  technician's 
cognitive  map  of  essential  physical  relations  (electronic,  hydraulic, 
electro-mechanical,  and  so  on)  in  a  complex  equipment  is  often 
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incomplete,  vague,  or  incorrect.  As  long  as  this  is  so,  any  series 
of  checks  and  test  readings,  though  apparently  well  motivated  and 
accomplished,  cannot  "close  in”  logically  on  a  faulty  unit.  All  the 
technician  can  do  in  this  environment  is  to  keep  making  checks  in  the 
hope  of  finding  some  drastic  test  indication,  or  to  engage  in 
probabilistic  or  mass  replacement  of  subunits.  Hie  conventional 
approach  has  been  to  provide  general  "theory"  training  in  electronics 
or  hydraulics  for  the  technician.  Armed  with  this  theory,  and  with 
the  assistance  of  technical  manuals,  the  technician  supposedly  could 
generate  his  own  fault-location  sequences.  But  modern  prime 
equipments  are  so  complicated  that  it  is  not  reasonable  to  expect  an 
ordinary  military  technician  to  know  them  well  enough  to  originate 
effective  test  sequences  for  himself,  after  a  few  months  of  training. 
It  is,  then,  not  so  much  the  technician's  reasoning  that  is  at  fault, 
as  it  is  his  information  base  regarding  the  meaning  of  the  checks  and 
test  readings  he  makes.  Given  a  sharp  map  or  model  of  the 
elimination  logic  that  prevails  between  observable  test  symptoms  and 
the  set  of  possible  troubles,  a  technician  can  usually  solve  a 
troubleshooting  problem  within  a  reasonable  time,  even  though  his 
logical  path  through  the  network  is  not  the  optimal  or  most  efficient 
sequence.  If  the  equipment  is  extremely  large  or  complex,  then  some 
kind  of  proceduralized  or  step-by-step  insructions  will  be 
necessary. 

It  follows  that  much  of  the  training  and  aiding  effort  should  be 
directed  to  preparing  a  special  symptom-malfunction  model  of  the  prime 
equipment,  and  making  that  model  accessible  to,  and  usable  by,  an 
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.  Right  now,  and 


for  the  next  few  years,  the  best  solution  probably  is  to  provide 
special  "aiding"  or  documentation  to  the  technicians,  to  have 
this  documentation  so  effective  that  it  will  actually  guide 
troubleshooting  down  to  the  appropriate  component  depth,  and  to  give 
the  technicians  intensive  practice  with  these  materials.  The 
competent  first  term  technician  thus  becomes  an  extremely  fluent  user 
of  complicated  booklets,  charts,  and  software  routines.  He  will  not 
often  originate  test  sequences  himself.  Rather,  he  will  follow  a 
definite  "tree  logic"  or  other  algorithm  that  will  guarantee 
localization  of  most  failures.  If  the  documentation  and  associated 
procedures  cannot  locate  certain  troubles,  then  the  aiding  package 
must  be  ixmediately  modified  so  that  it  will  accomplish  that  task  in 
the  future.  In  this  scenario,  we  do  not  lapse  into  the  conception  of 
"trained-ape  maintenance,"  where  an  untrained  person  blindly  follows  a 
conplex  test  routine,  and  does  what  it  tells  him  to  do.  (Xir 
technician  will  still  be  a  highly  skilled  person.  It  will  take  great 
care  and  judgment  on  his  part  to  carry  out  the  appropriate 
procedures.  The  aiding  technology  to  support  the  troubleshooter  is 
already  here,  and  has  been  proven  in  about  a  dozen  demonstration 
projects  (Shiver,  1975;  VanHeroel,  1979;  Foley,  1978) .  There  will 
always  be  a  need  for  military  maintenance  people  who  can  go  beyond 
the  procedural ized  aids,  and  can  solve  the  difficult  pcoblons.  These 
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people  will  have  to  possess  a  deeper  understanding  of  the  systems 
they  work  on;  perhaps  a  good  model  here  would  be  the  "flying  squad" 
specialists  in  the  computer  industry,  who  are  extremely  competent  in 
both  hardware  and  software  aspects  of  big  computers.  Training  such 
people,  and  providing  materials  for  them,  will  be  a  challenge  to  the 
military  personnel  community. 

Efficient  troubleshooting  can  be  achieved  in  the  military.  There 
are  many  successful  examples  from  Navy  submarines,  from  the  special 
weapons  area,  and  from  certain  aircraft  systems.  Also,  many  good 
people  in  the  military  system  think  that  they  are  helping  the 
troubleshooter.  Examples  of  these  are  the  people  who  originate  fault- 
location  schemes,  the  school  instructors,  and  the  manual  writers. 
According  to  our  view,  many  of  their  efforts  are  far  less  effective 
than  they  could  be,  and  this  is  because  they  do  not  stay  close  to  the 
actual  cognitive  situation  faced  by  the  military  technician.  One 
goal  of  this  report,  then,  is  to  explicate  seme  of  the  ways  that 
improvements  can  be  made. 

Bibliographical  Note 

The  troubleshooting  literature  is  widely  scattered  in  about  half 
a  dozen  major  clusters.  For  behaviorally  oriented  studies,  Souse's 
privately-circulated  bibliography  is  the  best  (Rouse,  1979).  In 
fact,  Rouse's  list  of  references  in  one  of  his  articles  constitutes  a 
reasonably  good  library  on  the  subject  (Rouse,  1978).  Johnson's 
(1972)  text  is  the  standard  reference  on  the  psychology  of 
reasoning.  Fault-tree  methodology,  probabilistic  troubleshooting, 


and  the  associated  mathenatics  are  sumarized  in  the  ONR-eponsored 
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report  on  reliability  and  fault-tree  analysis  (Barlcw,  et  al,  1975) , 
and  in  the  Ross  (1970)  textbook.  For  maintainability  program 
management,  the  best  general  source  is  still  the  Blanchard  and  Lowery 
(1969)  book;  although  the  microcircuitry  revolution  has  dated  some 
of  it,  this  text  has  thorough  treatments  of  maintainability 
prediction,  tradeoffs,  and  demonstration  procedures.  There  are 
several  new  books  on  the  logic  of  digital  troubleshooting  (Gasper ini, 
1979;  Coffron,  1979) .  The  computer-science  approach  to  complex 
search  problems  is  treated  by  Raphael  (1976) .  There  are  many  guides 
which  will  tell  how  to  design  a  maintainable  device,  a  good  example 
being  the  one  by  Morgan  et  al.,  (1963) .  State-of-the-art  in  military 
maintenance  aiding  and  training  is  well  reviewed  in  the  NTBC-Orlando 
conference  proceedings  (OTEJC,  1975;  NTBC,  1979),  and  in  Foley's 
sunmary  report  to  the  Air  Force  (Foley,  1978),  Foley's  summary  is 
especially  important,  because  it  lists  every  experimental  evaluation 
of  maintenance  aiding  up  to  1977,  and  it  also  provides  strong 
ackninistrative  advice  for  implementing  fully-proceduralized  aiding 
systems. 


SECTION  II.  LOCATING  A  FAILED  COMPONENT 

The  Loolc  of  Diagnosis 

Much  troubleshooting  is  very  sinple.  When  something  goes  wrong, 
all  or  aost  of  the  possible  causes  of  the  fault  are  listed,  and  are 
then  successively  eliminated.  The  process  is  continued  until  only 
one  culprit  remains.  That  item  is  repaired  or  replaced  and  the 
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problem  is  solved.  Or  maybe  one  component  is  known  to  be  likely  to 
fail,  and  it  is  therefore  summarily  replaced,  without  much  of  an 
attempt  at  fault-tracing.  Only  if  this  "obvious*  replacement  fails 
to  clear  the  trouble  is  any  further  search  undertaken.  Such 
troubleshooting  is  applied  in  all  practical  trades  and  professions, 
and  in  daily  life.  It  seems  so  natural  that  it  may  not  even  be 
formulated  in  an  explicit  way.  Technical  difficulties  arise,  of 
course,  when  the  list  of  possible  causes  is  incomplete,  when  it  is 
impossible  to  eliminate  some  possible  causes,  or  when  there  are  a 
great  many  related  "partial  causes."  But  the  oomnon- sense  conception 
of  trouble-isolation  stands  up  very  well,  and  is  especially  effective 
in  easily  visualized  physical  systems,  such  as  a  plumbing  layout. 

When  systems  get  more  complex  and  the  number  of  alternatives 
becomes  large,  a  simple  enumerate- and -eliminate  strategy  will  not  be 
practical.  It  was  just  barely  practical  for  repairing  TV  sets  in  the 
vacuum-tube  era.  As  a  rule  of  thunb,  replacing  tubes  would  repair  a 
malfunctioning  set  in  about  eight  times  out  of  ten.  But  a  big  air- 
control  radar  may  have  thousands  of  components  in  it,  with  many  of 
these  linked  together  in  complex  ways.  Furthermore,  there  is  good 
reason  to  think  that,  if  a  technician  attempted  simply  to  replace 
every  part,  he  would  introduce  far  more  troubles  than  he  would 
eliminate.  Evidently,  more  systematic  and  efficient  search 
strategies  are  needed, 
mimetic  Model g 

there  are  several  ways  to  display  the  relations  in  a  system: 
flow  charts,  schematic  diagrams,  and  so  an.  the  essentials  often  boil 


down  to  some  kind  of  logical  graph  or  table.  In  Figure  1,  a  graph  is 
shown  in  la;  it  has  four  vertices  (A,  B,  C,  D)  and  ten  arcs  (AB,  AD, 
BB,  etc.) .  The  relations  in  this  graph  nay  also  be  described  by  the 
Boolean  logic  matrix  in  lb,  or  by  the  dictionary  of  successors  shown 
in  lc  (Kaufmann  et  al.,  1977) .  The  "successor"  table  reproduces  the 
Boolean  matrix  row  by  row.  As  shown,  the  vertices  and  arcs  could  be 
anything  that  is  well-defined,  and  so  they  could  represent  nodes  and 
signal  flows  in  an  electrical  network.  Many  theorems  have  been 
derived  for  handling  the  relations  in  a  graph  or  logical  matrix.  One 
of  the  most  important  of  these  is  that,  for  each  state  of  the  set  of 
system  components,  a  unique  structure  function  can  be  defined.  Thus, 
one  can  calculate  the  influence  of  each  component  state  on  the 
overall  system  (Kaufmann,  1977;  Barlow,  Fussell,  &  Singpurwala, 

1975) .  As  another  and  more  surprising  example,  if  a  set  of  K  "links" 
of  a  monotone  function  are  known,  an  equivalent  network  can  be 
obtained  by  placing  in  parallel  K  subnetworks,  each  formed  of  the 
conponents  of  a  link  placed  in  series.  The  networks  obtained  in  this 
manner  can  also  be  "reduced”  to  their  simplest  form.  As  far  as  we 
know,  such  transformations  are  only  employed  in  theoretical  analysis 
of  systems,  and  are  never  utilized  by  working  technicians,  even  though 
such  analyses  do  express  all  the  logic  in  a  system. 
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Figure  1.  (a)  Graphical  representation;  (b)  Boolean 

matrix;  (c)  dictionary  of  successors. 

Hie  Boolean-table  idea  has  been  extended  to  handle  pseudo-verbal 
content,  and  to  print  out  "sentence  solutions"  to  computerized  search 
problems.  Findler's  Universal  Verbal  Puzzle  Solver  for  example, 
solves  any  well-formed  verbal  membership  puzzle  which  can  be 
expressed  in  terms  of  logical  inclusion,  exclusion,  and 
conditionality.  If  a  string  of  problem  sentences  is  inserted  the 
logical  relations  from  the  sentences  are  stored  as  a  long  list  of 
inclusions  and  exclusions.  When  all  are  entered  in  the  computer,  the 
program  does  a  brute-force  recursive  search  until  a  suitable  solution 
is  found  (Findler  et  al.,  1973).  Hie  search  loop  really  operates  upon 
the  equivalent  of  a  Boolean  matrix.  Though  it  has  apparently  never 
been  used  in  the  troubleshooting  domain,  the  Findler  program  would 
certainly  find  troubles  if  enough  logic  from  trouble-test  relations 
were  entered  in  a  pseudo-verbal  format.  The  technician  would  enter 
test  results  in  a  oonstrained-Ehglish  format,  until  isolation  was 
attained.  Another  kind  of  "logical  machine"  is  the  theorem-prover 
concept,  as  developed  by  Wang  and  others.  A  theorem-prover  program 
accepts  a  list  of  logical  premises  or  conditions.  Then,  to  search 
for  an  answer  to  a  puzzle  (or  a  trouble-shooting  problem) ,  one  asks 
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the  program  whether  a  given  "theorem"  (logical  statement)  is  valid. 

An  ingenious  reduction  routine  collapses  the  prenises  in  such  a  way 
as  to  permit  a  validity  check  on  any  well-formed  proposition 
involving  the  premises.  For  a  verbal  puzzle,  a  test  theorem  might  be 
"the  butler  murdered  the  viscount",  but  as  far  as  logic  is  concerned, 
a  theorem  could  be  "operational  amplifier  tl6  is  malfunctioning" 
(Raphael,  1976) .  To  be  useful,  both  the  Findler  and  Wang  concepts 
have  to  be  "full  of  data",  otherwise  no  solution  will  be  attained,  or 
else  the  logic  is  so  loose  that  a  number  of  pseudo-solutions  will  be 
printed  out,  with  the  machine  unable  to  discriminate  among  then. 

In  practical  maintenance  work,  a  useful  format  is  often  same 
kind  of  a  symptom-malfunction  (S-M)  matrix.  Many  projects  have 
discovered,  rediscovered,  and  formatted  the  basic  idea,  which  is 
nothing  more  than  a  table  showing  the  dependencies  among  symptoms  and 
components.  An  early  realization  of  this  was  in  the  BAMAGAT  project 
at  Hughes  Aircraft  in  the  1950's.  The  charts  used  there,  called 
maintenance  dependency  charts  (NX) ,  showed  an  ordered  list  of 
procedures,  a  rather  complete  listing  of  test  points  and 
measurements,  and  also  all  fwctional  units  that  generate  inputs  to  a 
given  test.  If  a  test  failed,  then  (hopefully)  only  a  limited  set  of 
hardware  signal  sources  had  to  be  checked  further.  That  basic  concept 
has  survived  several  renamings  as  SIN!  (Synbolic  integrated 
Maintenance  Manual)  and  FGM4  (Functionally  Oriented  Maintenance 
Manual) .  Boeder  and  Ranc  (1975)  believe  that  FGNM  is  most  successful 
with  straight-line  analog  functions,  and  that  logic  diagrams,  *ate 
tables,  or  trees  are  preferred  for  digital  circuitry.  In  the 
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simplest  non- sequential  format  of  S-M  matrix,  tests  are  often  listed 
as  rows,  with  components  or  subunits  in  the  many  columns.  As  the 
trouble  search  proceeds,  parts  of  the  table  are  logically  eliminated, 
and  when  one  or  a  few  units  are  left,  those  are  repaired  or 
replaced.  If  the  chart  is  correct  and  inclusive,  and  if  the  readings 
are  correctly  made  and  interpreted,  then  trouble  isolation  proceeds 
steadily.  Ihe  logic  in  such  charts  is  generally  easy  to  follow,  but 
in  the  practical  case  there  is  often  ambiguity  about  the  normality  of 
a  complex  signal  or  waveform,  and  so  if  a  mistake  is  made  the  search 
may  not  converge  (Joyce,  1975).  Almost  always,  though,  anmp  trouble 
isolation  will  be  achieved.  Most  research  people,  we  believe,  who 
work  closely  with  maintenance  people  in  the  field  have  come  to  favor 
aiding  the  technician  with  some  kind  of  S-M  matrix  approach.  The 
basic  idea  of  maintenance  dependency  also  can  be  enhanced  by 
following  good  display  and  presentation  features,  as  inaba  (1979)  and 
others  have  demonstrated  with  the  PINO  format.  If  the  system  is 
extremely  large,  then  almost  any  format  becomes  unwieldy.  Roeder  and 
Ranc  (1975)  estimated  that  for  one  system,  some  733  sheets  of  PCMM 
function  block  diagrams  would  be  required  to  describe  the  system! 

It  is  best  to  provide  good  S-M  information  to  the  field 
technician,  and  not  make  him  generate  it  for  himself.  The  general 
rationale  for  this  is  that  when  working  under  time  and  operational 
stresses,  the  technician  is  not  in  a  good  position  to  be  discovering 
logical  relations.  At  that  time,  he  should  be  ualno  effective 
relations  already  discovered  by  himself  and  by  others.  For 
large  digital  systmns,  there  is  a  further  rationale.  Many  IC  chips 


are  delivered  to  the  system  designer  with  data  sheets  which  describe 
chip  operation  in  terms  of  gate  equivalents.  But  the  data  sheet  may 
not  be  complete ,  or  when  it  is  encapsulated  into  a  system  the  unit 
can  experience  logical  ambiguities  caused  by  timing,  noise,  stray 
capacitance,  or  threshold  problems.  Often  the  only  way  to  separate 
all  these  causes  of  logical  failure  is  to  do  high-speed  test  runs  via 
automatic  test  equipment  (ATE) .  So  in  same  cases  the  technician 
could  not  possibly  use  logical-status  derivations  from  a  simple 
consideration  of  the  gating  logic. 

PLQbflbjlisUS  MfiaaslS 

Many  aspects  of  systems  are  probabilistic  in  character.  Life 
and  failure  expectations  for  components,  for  subsystems,  and  for  the 
whole  system  can  be  expressed  in  terms  of  probability  distributions. 
There  are  also  distributions  for  the  time  it  takes  to  set  up  and  make 
different  test  readings,  for  the  likelihood  that  a  particular  failure 
will  be  detected  when  it  occurs,  for  the  chance  that  a  troubleshooter 
will  introduce  a  new  trouble  as  he  works  on  the  prime  equipment,  and 
so  on.  As  all  these  distributions  can  change  during  the  course  of  a 
search,  analysts  have  studied  ways  of  structuring  and  optimizing  the 
process.  Powerful  solution  techniques  such  as  linear  programming  and 
dynamic  programming  can  be  brought  to  bear. 

A  simple  probabilistic  example  will  give  some  of  the  flavor. 
Consider  each  test  or  check  reading  as  a  means  of  reducing 
uncertainty  oonceming  the  location  of  a  fault.  In  the  language  of 
information  theory  (Guiasu,  1977) ,  the  mount  of  uncertainty  resolved 
by  a  test  t  is  given  by 
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1  6  Z. 

which  is  the  sum,  over  all  sets  of  modules  into  which  the  test 
divides  the  system  Z  (x) ,  of  the  probability  of  the  fault  being 
within  the  set,  p  (s. ) ,  times  the  logarithm  to  base  two  of  this 
probability.  Each  test  is  then  conceived  as  dividing  the  system  into 
two  subsets  (the  fault  being  in  one  or  the  other) .  The  probabilities 
in  the  above  expression  are  conditional  upon  all  test  information 
obtained  up  to  any  point  in  a  given  test  sequence. 

A  rational  or  optimal  technician  might  then  perform  next  that 
test  which  resolves  the  greatest  amount  of  uncertainty,  per  time 
spent  testing.  Thus,  he  should  maximize 


D  (T)  =  W  |Tj»  where  t  (x)  is  the 

time  required  to  perform  the  test,  starting  at  whatever  system  and 
test  condition  prevails  just  then. 

Towne's  dynamic  programing  (DP)  formulation  (Rigney  and  Towne, 
1977)  utilzes  the  structure  of  the  system  to  yield  deterministic 
relationships  between  indicators  (including  test  points)  and  the 
elements  which  they  monitor.  Time  costs  for  extracting  information 
from  each  indicator  are  then  either  estimated  or  computed.  These 
might  include  times  for  test  equipment  setup,  partial  disassembly,  or 
even  walking  to  another  location.  Finally,  reliabilities  are 
associated  with  each  hardware  element.  The  DP  algorithm  then 
successively  computes  the  optimum  next  "indicator"  to  sample,  in 
order  to  minimize  expected  (time)  cost.  This  technique  would  be 
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considered  probabilistic,  since  the  reliability  data  are 
probabilistic. 

Guiasu  (1977)  gives  a  similar  model  of  classifying  a  system  (in 
our  domain,  classifying  a  system  would  identify  the  trouble  state, 
i.e. ,  locate  the  trouble).  His  model  calculates  the  entropy  of  each 
test,  weights  it  by  the  cost  of  a  test  (again,  this  could  be  the  time 
or  inconvenience  required  to  perform  it) ,  and  then  selects  the  test 
that  has  the  maximum  weighted  entropy.  This  scheme  always  leads  to 
the  trouble,  with  the  smallest  expected  oost.  If  checks  involve 
extraordinary  amounts  of  time  or  risk  to  set  up,  such  an  analysis 
might  be  quite  practical.  One  application  used  the  model  to  select 
which  of  the  27  possible  clinical  tests  should  be  made  next  on  a 
human  patient,  to  diagnose  any  of  13  renal  diseases.  Using  the  model, 
an  average  of  about  eight  tests  are  required,  to  oomplete  the 
diagnosis,  instead  of  doing  the  full  set  of  27  tests. 

Models  like  these  are  certainly  elegant  rationales  for  test 
selection,  which  can  be  valuable  for  judging  the  effectiveness  of  a 
particular  strategy  and  for  developing  procedural  guides.  It  is 
unlikely,  however,  that  a  technician  oould  employ  these  techniques  to 
generate  his  own  search  process.  There  may  be  some  rough  heuristics 
that  can  be  applied  by  technicians,  which  come  out  of  such  models. 

The  information  theory  approach  suggests,  for  example,  that  if  all 
tests  are  reasonably  close  to  splitting  the  remaining  sets  of 
candidates  into  even  halves,  the  next  test  should  be  the  one  that 
takes  the  shortest  tine.  Also,  if  all  tests  are  long  shots  (less 
than  5  percent  chance  of  finding  the  fault) ,  then  the  test  that  takes 
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the  least  time  often  should  be  selected.  Such  rules  might  already  be 
known  fairly  well  by  the  working  technician. 

Human  search  Behavior 

As  mentioned  earlier,  it  has  been  known  for  twenty-five  years 
that  a  symbolic  format  can  capture  much  of  troubleshooting  behavior, 
and  can  get  at  the  crux  of  the  search  task,  This  was  shown  with 
various  kinds  of  devices:  tab  tests,  masonite  boards  with  little 
windows,  punchboards,  and  special  paper  formats  (Grings,  1953;  Glaser 
et  al.,  1952;  Fattu,  1956).  To  be  sure,  the  synthetic  format  leaves 
out  such  key  behaviors  as  locating  test  points,  connecting  meters, 
and  being  generally  careful  and  safe  around  dangerous  hardware. 

Unless  the  technician  can  troubleshoot  effectively  at  the  symbolic 
level,  however,  it  would  be  of  little  avail  to  be  skilled  in  these 
other  aspects  of  troubleshooting.  Most  investigators  have  therefore 
used  some  kind  of  synthetic  format  for  their  troubleshooting  studies. 

Nearly  every  study  of  troubleshooting  behavior  has  noticed  great 
variation  in  the  search  sequences  taken  by  technicians.  Even  for 
rather  simple  electronics  circuits,  exact  replications  of  paths  are 
rarely  found.  When  detailed  records  of  performances  are  examined  no 
more  than  about  half  of  the  moves  "make  sense"  in  retrospect  (Bryan 
et  al.,  1956).  Why  do  we  see  there  such  marked  individual 
differences,  so  much  uninterpretable  behavior,  and  so  much  apparent 
"wandering?”  The  best  answer,  we  believe,  is  that  ordinary 
technicians  differ  in  their  knowledge  about  the  normal  system  and  in 
the  significance  they  attach  to  readings  in  the  abnormal  system. 

They  simply  do  not  share  the  same  information  bases,  despite 
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similarities  of  training  and  experience.  This  can  be  quickly 
demonstrated  at  any  military  installation  by  presenting  a  few 
technicians  with  a  standard  transistor  amplifier  schematic,  and 
asking  them  separately  what  changes  in  the  currents  and  voltages 
would  result  from  parametric  changes  in  one  of  the  parts,  such  as  a 
resistor  that  increased  in  value.  There  will  not  be  much  agreement, 
and  many  mistakes  will  be  noted. 

It  is  likely,  then,  that  most  technicians  simply  do  not  remember 
much  of  the  electronics  theory  that  has  been  presented  to  them  in 
schools  and  bodes.  One  study  done  in  the  early  1960's  at  a  Navy 
repair  facility  presented  simple  test  items  involving  the  "fifteen 
basic  vacuum-tube  circuits"  to  a  sample  of  civilian  shipyard 
technicians,  along  with  some  elementary  electronics  theory  such  as 
Kirchhoff's  Laws.  The  goal  was  to  find  out  which  circuit  principles 
were  most  difficult  for  the  men,  and  how  these  correlated  with 
knowledge  of  basic  theory.  All  the  scores  were  so  low  that  the 
research  planned  on  those  subjects  could  not  proceed.  In  fact,  it  was 
unclear  how  these  men  could  understand,  or  could  repair,  the  complex 
radars  and  radios  they  worked  on.  One  possibility  is  that,  although 
they  could  not  manage  detailed  circuit  analysis,  they  could  perform  a 
rather  cut-and-dried  overhaul  and  checkout  procedure.  When 
discussing  these  results,  the  supervisors  explained  that  detailed 
circuit-analyzLig  skills  were  useful  for  designers,  but  not 
necessarily  required  for  maintenance  shop  people.  A  supervisor  would 
often  say,  "I  don't  know  that  kind  of  stuff  nyself ,  now."  Twenty 
years  ago,  Williams  and  Whitmore  (1959)  showed  that,  whereas 
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knowledge  of  electronics  theory  about  the  Nike-Ajax  missiles  system 
was  highest  at  the  end  of  the  training  school,  ability  to  find  faults 
was  poor.  Then,  fault-finding  ability  improved  with  field  experience, 
but  knowledge  of  theory  declined.  Shepherd  et  al,  (1977)  think  that 
it  is  possible  that  technicians  may  develop  rough  and  effective 
search  rules  during  the  field  work  (and  may  attempt  to  use  theory 
while  appreciating  these  rules) ,  but  once  the  rules  are  in  hand, 
theory  can  be  forgotten.  Foley's  (1974)  review  suggested  that  the 
correlation  between  theory  test  scores  and  performance  is  low,  as  is 
the  relation  between  school  marks  and  job  performance.  Thus, 
"teaching  theory"  is  doubly  ineffective.  People  do  not  learn  or 
retain  it  well,  and  even  if  they  did,  such  knowledge  is  not  strongly 
tied  to  maintenance  effectiveness. 

Perhaps  the  most  famous  behavioral  anal sy is  of  troubleshooting 
was  Miller's  investigation  of  the  "half-split"  technique  (Miller  et 
al,  1953) .  The  half-split  procedure  reconmends  that  a  chain  of 
system  elements  be  checked  first  in  the  middle  of  the  chain,  or  in 
such  a  way  that  the  two  groups  of  elements  on  both  sides  are  equally 
likely  to  contain  the  faulty  element.  The  half-split  idea  has  been 
extended  and  generalized  in  various  ways,  to  permit  inclusion  of 
parallel  paths,  probability-of-failure  data,  and  cost  figures.  While 
people  can  learn  the  half-split  idea  (Hannom  et  al.r  1967),  it  is 
difficult  to  employ  except  in  the  most  crude  and  obvious  decision 
situations,  such  as  a  string  of  lights.  In  difficult  real-world 
problems,  units  and  subsystems  of  widely  different  reliabilities  are 
arranged  in  complex  and  non-linear  chains,  with  obscure  barriers  and 
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feedbacks  between  links  in  the  chain.  Otherwise,  troubleshooting 
would  be  rapid  and  conclusive  by  any  reasonable  method,  and  the 
slight  increases  in  efficiency  caused  by  half-splitting  would  be 
negligible. 

Other  optimal  search  models  can  also  be  formulated,  and  their 
troubleshooting  effectiveness  can  be  compared  with  actual  human 
behavior.  As  an  example  of  this,  a  computerized  Bayesian  search 
policy  was  defined  for  a  certain  symptom-malfunction  matrix  (Bond  & 

Rigney,  1966) .  Hie  computer  program  operated  on  the  conditionals  that 
obtained  between  test  symptoms  and  component  parts.  When  a 
malfunction  was  introduced,  the  program  would  always  find  the  trouble 
in  the  smallest  number  of  steps  or  checks  (if  the  trouble  could  be 
found  from  the  data  given) .  By  updating  the  failure-likelihood 
estimates  for  each  component  after  each  test,  the  program  acted  as  a 
good  statistician  would,  if  confronted  with  a  fault-location 
problem.  When  a  human  technician's  search  record  is  compared  to  the 
performance  of  such  a  program,  it  is  always  found  that  the  human  is 
not  an  optimal  troubleshooter.  He  makes  redundant  checks,  forgets 
things  he  has  already  done,  replaces  components  which  logically 
cannot  be  faulty,  and  often  seems  to  mill  around  the  prime  equipment 
in  a  near-aimless  way.  One  study  estimated  human  search  efficiency 
at  about  thiry  percent  of  the  optimal  (Rigney,  1968) .  Such 
observations  might  suggest  that  technicians  are  illogical  or  feckless 
in  their  work,  which  is  not  so.  The  situation  resembles  that  which 

t 

is  found  in  the  psychology  of  reasoning.  When  intelligent  people  are  given 
syllogistic  problems  to  solve,  many  errors  and  inoorrect  inferences 
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will  be  found.  On  close  examination,  however,  many  of  these  mistakes 
are  largely  due  to  the  artificiality  of  the  syllogism,  and  to  the 
subject's  unfamiliarity  with  the  way  that  English  terms  like  "or"  and 
•some"  are  used  in  logic  (Johnson,  1972) .  When  corrected  and 
practiced,  and  especially  when  syllogisms  are  cast  in  a  familiar  or 
"real"  context,.. subjects  are  indeed  quite  logical.  Going  back  to  the 
troubleshooting  context,  the  frequent  near  randomness  of  a  search 
pattern  is  largely  due  to  a  vague  and  incomplete  conception  of  the 
basic  system,  and  of  the  measurements  made  upon  that  system.  An 
additional  bit  of  evidence  on  this  point  comes  from  the  Bayesian 
fault-locator  study  mentioned  above  (Rigney,  1968) .  In  an  extension 
of  that  research,  technicians  were  required  to  construct  their  own 
subjective  symptom-malfunction  matrices.  To  do  this,  they  entered 
their  estimates  of  the  relationship  between  a  reading  at  a  test  point 
and  a  possible  malfunction  in  each  component  cell.  Then,  taking 
these  subjective  estimates  as  probabilities,  the  Bayesian  comparison 
routine  was  run  again.  It  was  learned  that  while  most  technicians  are 
not  "good  Bayesians,"  even  on  their  own  subjective  matrices,  their 
search  does  proceed  rather  well,  if  inefficiently,  and  would  locate 
troubles  eventually,  if  ihfi  matrices  were  correct  enough.  Again  the 
conclusion  is  rather  strong:  it  is  the  information  base,  rather  than 
the  search  algorithm,  or  search  efficiency,  that  best  predicts  success. 

For  an  overall  picture  of  technician  activities.  Table  1  gives  a 

% 

breakdown  from  one  set  of  experimental  observations  (Rigney  et  al. , 
1968) .  The  table  is  fairly  discouraging.  Host  of  the  time  (58.7%) , 
these  troubleshooters  did  not  have  the  malfunctioning  unit  included 
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in  their  set  of  "possibles."  About  10  per  cent  of  the  time,  the 
equipment  was  classified  as  fully  operational,  even  though  the 
malfunction  persisted;  and  about  13  percent  of  the  time  the  trouble 
already  had  been  repaired,  but  the  technicians  dich't  recognize  that 
fact. 


Table  1 

Technician's  State 

TotalTime 

(min.) 

%  of  Total 

Perfect  hypothesis  set 

52.3 

1.3 

Malfunction  properly  suspected, 
but  hypotheses  imperfect 

666.3 

16.5 

Malfunction  not  included  in 
hypotheses 

2,278.8 

58.7 

Equipment  judged  all  right 
despite  malfunction 

405.0 

10.0 

Malfunction  actually  repaired 
but  not  known 

546.0 

13.5 

A  recent  program  of  research  by  Rouse  (1978)  has  been  directed  to 
hunan  search  behavior  in  digital  networks.  The  subject  faces  an 
arrangement  of  AND  gates,  some  of  which  are  good,  and  same  of  which 
are  not  operating.  As  in  the  Bayesian  search  comparison  mentioned 
previously,  subjects  did  not  perform  optimally,  but  they  were  superior 
to  a  brute-force  searching  scheme,  and  they  could  be  aided  by  special 
software.  When  network  size  was  increased,  performance  deviated 
further  from  optimal.  Two  incidental  results  from  Rouse's  work  are 
intriguing  to  the  psychologist.  One  of  these  findings  was  that  self- 
paoed  initial  training  can  transfer  positively  to  a  forced-paoed  test 
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situation.  Another  discovery  was  that  the  unaided  subjects  did  not 
seen  to  appreciate  the  diagnostic  value  of  a  "good"  test  at  a  node 
(node  not  failed) .  This  observation  is  similar  to  Wason  &  Laird's 
evidence  on  the  modus  tollens  fallacy  in  human  reasoning,  wherein 
subjects  also  did  not  realize  the  meaning  of  information  that  did  not 
directly  support  the  hypothesis  under  review  (Wason  &  Laird,  1968) , 
Ihus, 

"In  our  fault  diagnostic  context,  this  idea  means  that  humans 

have  difficulty  in  effectively  utilizing  a  test  that  indicates 

some  component  has  not  failed  (Rouse,  1978) . 

This  is  one  instance  where  a  troubleshooting  experiment  can  be  inte¬ 
grated  nicely  with  cognitive  psychology. 

Rasmussen  and  Jensen  (1974)  watched  Danish  technicians  working  on 
real  equipment  in  atomic  power  plants.  They  noticed  that  technicians 
had  a  tendency  to  "do  something,"  rather  than  to  "reason  it  out," 
hence  troubleshooting  performances  often  look  busy  but  inefficient, 
when  gauged  by  optimal- test  standards.  As  many  other  researchers 
have  noted,  there  were  frequent  "....rapid  sequences  of  good-bad 
checks  on  actual  signals  against  normal  signals."  Most  of  the 
observed  performances  oould  be  classified  as  being  either 
topographic,  functional  search,  or  evaluation  of  a  fault.  In  a 
topographic  troubleshooting  attempt,  the  technician  may  "lose"  a  good 
signal  in,  say,  an  amplifier.  The  amplifier  itself,  and  the  paths  in 
and  out  of  it,  would  then  be  explored.  Functional  search  proceeds  on 
a  "failed  function"  basis.  Thus,  if  a  TV  picture  is  low,  one  might 
consider  first  the  circuits  that  control  the  vertical  deflection 
plates.  A  person  who  evaluates  a  fault  derives  information  from  the 
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fault  itself,  without  explicit  localization.  A  "fuzzy"  video  signal  is 
often  due  to  ripple  in  the  main  supply,  and  so  the  voltage  regulators 
may  not  be  doing  their  job,  when  this  symptom  appears. 

If  one  of  Rasmussen's  technicians  was  stuirped,  he  might  pause, 
and  then  during  the  break  in  performance  have  some  fresh  ideas  about 
things  to  do.  At  several  places  in  the  analyses,  the  disparity 
between  "engineering  thinking"  and  technician  work  methods  was 
evident.  The  engineer  often  wants  an  intellectual  understanding  of 
the  trouble,  and  a  rationale  for  its  associated  parametric  changes 
throughout  the  hardware.  On  the  other  hand,  the  technician's  goal  is 
to  find  this  trouble,  and  considerations  of  the  logic  or  elegance  of 
the  solution  are  secondary.  Dus  is  not  to  say  that  technicians 
cannot  adnire  an  ingenious  search  scheme,  but  their  focus  is  usually 
quite  practical,  and  redundancy  does  not  bother  them  at  all. 

Wescourt  and  Hemphill  (1978)  studied  computer  program  debugging 
in  both  expert  and  novice  programmers.  The  expert  may  indeed  have  a 
better  general  strategy  than  the  novice,  but  his  most  important 
capability  is  specialized  memory  for  certain  program  behavior 
characteristics,  that  is,  he  has  a  large  "library"  of  experience  which 
often  enables  him  to  bypass  the  usual  deductive  processes.  A  novel 
feature  of  the  Wescourt  investigation  was  the  explicit  recognition 
that  advanced  programmers  often  knowingly  write  incorrect  programs, 
finding  it  easier  to  debug  than  to  perform  the  logical  analysis 
necessary  for  correct  ooding. 

A  great  limitation  of  much  behavioral  research  in  the 
electronics  troubleshooting  area  is  that  very  s inpie  circuits  and 
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equipment  are  employed  as  the  performance  vehicles.  With  the 

exception  of  the  Rasmussen  study  and  a  few  others,  the  academic 

researchers  have  stayed  with  simple  DC  circuits,  Ohm's  Law,  and 

similar  engineering  rudiments  that  are  easily  analyzed.  The  people 

who  have  really  faced  up  to  the  big  prime-equipment  problems  are  j 

those  who  were  under  contract  to  produce  a  specific  aid  or  training 

system,  and  to  demonstrate  it  on  an  item  of  operational  complexity. 

Actually,  major  troubles  may  require  a  very  lengthy  analysis.  Anyone 
who  does  not  believe  this  should  consult  Coffron's  account  of  an 
advanced  troubleshooting  problem  in  a  system  made  up  of  standard  IC 
chips.  Some  thirty  pages  of  technical  discussion  are  required  to 
explicate  the  test  rationale  (Coffron,  1979).  Even  a  procedural 
guide  to  this  task  would  take  several  pieces  of  paper. 

Another  big  gap  in  behavioral  research  on  troubleshooting  is  the 
near-absence  of  data  on  just  how  the  technician  codes,  remembers,  and 
retrieves  information  about  the  prime  equipment.  Researchers  often 
focus  too  much  on  the  outcome  of  the  performance,  such  as  the  time  to 
solution,  or  success  or  failure  in  finding  a  trouble,  when  they 
should  be  watching  the  process.  Electronics  material  is  abstract, 
and  that  is  one  reason  why  it  is  so  difficult  to  leam.  What  imagery 
and  analogies  can  be  used  in  visualizing  the  relationships  in  an  AC 
circuit?  There  are  some  hints  from  mnemonics,  and  we  know  that 
making  abstract  material  meaningful,  or  coding  items  into  a  story 
line  can  facilitate  memory,  but  do  subjects  use  such  techniques?  And 
could  they  easily  leam  to  use  them?  we  know  that  years  of  intensive 
practice,  say  in  engineering  school,  can  produce  great  fluency  in 
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certain  basic  circuit  relations,  but  we  have  few  tricks  to  facilitate 

or  to  short-cut  such  long  exposure.  There  are  learning  difficulties, 

too,  in  going  from  qualitative  to  quantitative  appreciation  of 

relations.  Suppose  you  are  displaying  the  several  currents  and 

voltages  in  an  amplifier  stage.  Each  little  current  can  be  shown  in  a 

different  color,  and  these  currents  can  then  be  discussed  separately 

under  the  operating  conditions  of  the  circuit,  as  in  the  remarkable 

Adams  series  of  textbooks  (Adams,  1966) .  If  a  component  has  changed 

slightly  in  value,  though,  the  troubleshooter  eventually  may  have  to 

perform  quantitative  analysis,  perhaps  via  sets  of  equations.  How 

does  this  bridging  from  the  qualitative  to  the  quantitative  come 

about?  Again,  what  imagery  can  support  the  performer?  In  other 

settings,  Bruner  (1965)  has  shown  how  ingenious  and  concrete  imagery 

schemes  can  make  abstract  and  difficult  concepts  accessible  to 

children.  For  instance,  children  can  appreciate  the  concept  of  the 

energy  quantum.  We  need  some  of  this  ingenuity  to  support  troubleshooter 

cognition.  There  is  some  evidence  that  the  ability  to  bring  up  vivid 

voluntary  images  can  aid  in  problem  solving,  through  its  association 

with  "independence  of  judgement"  (Richardson,  1969) .  A  related 

cognitive  hint  comes  from  Rowan  (1965) ,  who  tested  a  programmatic 

approach  to  solving  algebra  word  problems: 

"...The  program  'translates'  the  word  problems,  step  by  step 
into  algebraic  equations.  A  number  of  algebra  students  do 
the  same.  But  other  youngsters  first  translate  the  English 
prose  into  a  'picture'  of  the  physical  situation  and  then 
translate  this  representation  into  equations.  And  those  who 
perform  this  indirect  rather  than  direct  translation  prove 
to  he  more  powerful  problem  solvers." 

This  conclusion  might  apply  directly  to  the  circuit  analysis  domain; 
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good  circuit  analysts  should  be  those  who  can  originate  a  clear  picture 
of  circuit  actions. 

When  subjects  are  presented  with  a  logically  simple  trouble¬ 
shooting  problem,  and  can  approach  it  from  either  a  "work  saving"  or 
"probability"  standpoint,  they  initially  perform  easy  stage  checks, 
gradually  shifting  to  the  more  likely  but  more  difficult  kinds  of 
testing.  At  least  they  did  so  in  the  Detambel  and  Stolurow  (1957) 
experiment,  when  work  was  represented  by  the  number  of  screws  in  the 
lid  of  each  component,  and  probability  was  the  likelihood  of  a 
malfunction.  As  in  the  other  work  cited,  there  were  few  optimal 
troubleshooters.  A  similar  setup,  but  with  more  units,  was  used  by 
Dale  (1959) .  His  display  had  nineteen  electric  sockets,  one  of  which 
was  defective.  As  far  as  the  subjects  knew,  there  was  no  particular 
arrangement  between  the  sockets.  About  a  third  searched  randomly, 
but  some  used  a  stepwise  pattern.  When  the  subjects  were  told  that 
the  current  was  flowing  through  the  nineteen  stages,  and  that  the 
malfunction  oould  interrupt  the  current  flow  at  any  point,  subjects 
were  more  systematic.  As  expected,  the  more  informed  subjects  were 
more  efficient. 

Neimark  (1967)  used  a  problen  board  equipped  with  shutters  which 
could  control  viewing  of  test  information.  The  optimal  half-split 
strategy  showed  up  again  here,  although  few  people  applied  it 
strictly.  Sane  subjects  uncovered  good  information  on  the  first 
look,  and  therefore  tended  to  become  "gamblers.”  These  people  would 
eventually  shift  to  a  more  systematic  and  conservative  scheme, 
however,  if  their  gambling  did  not  pay  off.  The  results  again 
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indicate  a 'general  rationality,  but  not  optimality,  in  search 
behavior. 

For  some  complex  tasks,  knowledge  hiearachies  can  be  defined. 
Gagne  et  al.,  (1961)  took  as  the  terminal  behavior  the  capability  of 
finding  formulas  for  the  sum  of  n  terms  in  a  number  series.  This 
terminal  behavior  was  broken  down  into  prerequisite  behaviors,  and 
these  were  then  arranged  in  an  ascending  series,  which  were  taught 
one  level  at  a  time.  The  fined  or  highest  level  was  not  attempted 
until  all  the  prerequisite  behaviors  were  mastered.  Most  of  the 
empirical  studies  of  this  learning-hierarchy  approach  use  "clean" 
mathematical  tasks,  and  in  those  cases  the  data  are  predicted  well 
from  the  strati fied-skill  arrangement  (Gagne  et  al.,  (1961).  While 
most  training  courses  do  assume  prerequisites  and  cumulative 
knowledges,  a  logically  engineered  hiearachy  does  not  guarantee 
learning  efficiency,  and  may  even  hold  up  progress  (Merrill,  1965? 
Johnson,  1972).  Subjects  can  learn  "from  the  top  down,"  and  can 
master  lower-level  skills  while  working  on  the  higher  levels.  It  is 
still  true,  of  course,  that  at  some  point  the  lower-level 
capabilities  will  have  to  be  developed.  The  learning  hierarchy  idea 
has  not  been  fully  been  evaluated  in  troubleshooting  of  real 
equipment,  though  hierarchies  were  defined  for  a  fairly  involved  DC 
circuit  problem  (Wollmer  &  Bond,  1975) . 

Cognitive  styles  can  sometimes  predict  aspects  of  intellectual 
performance  that  cannot  be  explained  by  aptitude  and  experience.  One 
style  feature  is  the  way  that  a  problon  solver  gathers  information 
about  his  problem.  In  Gaier's  (1955)  Air  Force  study,  trainees  who 
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preferred  to  seek  principles  rather  than  facts  performed  better  in  a 
mechanics  course.  Gardner  &  Schoen  (1962)  found  a  stable  style  of 
cognitive  control  in  their  free-sorting  test  situation.  Some 
subjects  made  sharp  and  genuine  cognitive  differentiations  when 
arranging  a  set  of  items,  while  others  merely  listed  concrete 
features,  and  did  little  "real  thinking."  Interestingly,  this 
"differentiating"  cognitive  style  was  independent  of  general 
intelligence.  Perhaps  it  could  be  confirmed  in  a  troubleshooting 
donain,  and  could  help  to  explain  the  wide  individual  differences  in 
eventual  performance,  among  technicians  who  are  all  selected  on 
general  intelligence.  Similar  expectations  might  apply  to  the  Witkin 
"field  independence"  concept,  since  people  scoring  high  on  that 
variable  tend  to  achieve  figure  concepts  readily,  and  to  perform  well 
on  the  classical  problems  of  Duncker  and  Maier  (Davis  &  Klausmeier, 

1970) .  Nearly  all  of  the  cognitive  style  variables  proposed  so  far 
are  of  questionable  utility,  insofar  as  reliability  and 
general izability  Jure  concerned,  but  they  offer  some  intriguing 
research  possibilities. 

As  mentioned  earlier,  negative  information  is  not  employed  in  the 
same  way  as  positive  information.  Wason's  (1960)  research  is  a  good 
illustration.  In  asking  college  students  to  generate  a  rule  from  a 
number  series,  he  instructed  ...."You  will  be  given  three  numbers 
which  conform  to  a  simple  rule  that  I  have  in  mind.... Your  aim  is  to 
discover  this  rule  by  writing  down  sets  of  three  numbers,  together 
with  reasons  for  your  choice.”  A  significant  finding  was  that  the 
subjects  seemed  to  avoid  negative  or  discontinuing  examples  that 
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could  eliminate  a  given  hypothesis.  Campbell's  (1965)  experiment  was 
along  the  same  line.  It  presented  word  problems  wherein  the  letters 
of  the  words  were  replaced  by  other  letters,  if  the  letter 
substitution  had  to  be  done  indirectly,  by  elimination,  then  the 
problems  were  more  difficult.  Johnson  (1972)  interprets  such  results 
in  terms  of  information  load: 


"....There  is  an  upper  limit  to  the  amount  of  information  that 
the  individual  can  handle,  and  that  as  he  approaches  this  limit 
he  makes  errors.  Presumably,  he  retains  the  information  that 
first  comes  to  him  while  taking  in  new  information  to  be 
related  to  it.  If  he  can  relate  the  new  information  directly 
to  what  is  being  retained,  solution  is  easy.  But  if  he  must 
transform  the  new  information,  as  from  negative  to  affirmative, 
or  reverse  the  order,  or  change  the  subject,  the  total  load  may 
approach  his  capacity  and  errors  may  occur." 

The  positive-negative  logical  disparity  in  human  processing  can 
also  be  demonstrated  by  timing  the  performance  on  specially- 
constructed  items.  The  statement  "seventy-eight  is  an  odd  number" 
can  be  answered  quicker  than  the  negative  counterpart  "fifty-seven  is 
not  an  even  number."  As  Johnson  (1972)  observes,  such  items  can  be 
written  in  completion  form,  and  again  the  positive  statement  is 
evaluated  sooner.  Jones  (1966)  showed  that  subjects  can  mark  "3,  4, 

7,  8"  faster  that  they  can  mark  "all  numbers  except  1,  2,  5,  6." 
Perhaps  the  ultimate  in  timing  technology  in  this  domain  was  achieved 
by  Tr abas so  (1967) .  He  gave  one  subject  32  selection  rules  about  two- 
part-figures  that  were  systematically  varied  in  size  (large  or 
small)  and  color  (orange  or  green) .  A  rule  was  phrased  in  terms  of 
logical  "and”  (conjunctive)  or  logical  "or"  (disjunctive) .  A  rule 
like  "orange  and  orange"  was  verified  faster  (.634  secs.)  than 
"orange  and  not-green"  (.736  secs.) ,  and  "not  green  and  not  green" 


took  .824  seconds.  Trabasso  then  assumed  that  these  time  differences 
were  due  to  a  basic  transformation  time.  Thus,  "orange  and  not- 
green,"  though  logically  equivalent  to  "orange  and  orange,"  took 
about  a  tenth  of  a  second  longer  because  one  transformation  was 
required  for  the  not-green  term.  Two  transformations  ("not  green  and 
not-green")  would  add  another  tenth  of  a  second.  Perhaps  such 
transformation  times  are  basic  parameters  of  human  information 
processing  capability.  As  far  as  we  know,  nobody  has  measured  them 
in  troubleshooters. 

Conditional  reasoning  involves  some  kind  of  if-then  contin¬ 
gency.  Here  again,  many  logical  errors  are  observed  when  people  are 
asked  to  process  conditional  statements.  The  format  structure  can  be 
expressed  in  P  Q  form,  as  follows  (Johnson,  1972) : 

If  X  is  a  fish,  then  X  can  swim  P  — Q 

X  is  a  fish  P 

Therefore,  X  can  swim  Q 

Nearly  everybody  perceives  this  as  a  valid  argument.  But  the 
following  fallacy  is  also  perceived  as  valid: 

If  X  is  a  fish,  then  X  can  swim  P  — ►  Q 

X  can  &/im  Q 

Therefore,  X  is  a  fish  P 

One  trouble  with  the  conditioned  is  that  the  subject  may  put 
temporal  or  causal  meaning  into  a  structure  which  is  supposed  to  be 
strictly  logical.  English  words  like  if  and  then  have  a  distinct 
time  and  order  flavor  about  them.  Also,  the  same  difficulty  with 
handling  negation  in  ordinary  logic  is  found  with  conditional 
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falsity.  Wason's  (1968)  demonstration  involves  only  four  simple 
cards,  but  it  shows  the  logical  and  psychological  skeleton  of  the 
conditional.  The  four  cards  have  D,  3,  B,  and  7  on  their  respective 
faces,  and  the  subject  is  asked  to  evaluate  the  validity  of  the  rule: 
if  there  is  a  D  on  one  side  of  any  card,  there  is  a  3  on  the  other 
side.  Which  cards  should  be  turned  over?  The  correct  answer  is  D 
and  7.  In  the  event,  everybody  selected  D,  because  if  there  was  a  3 
on  the  other  side  the  rule  would  be  violated,  but  it  was  much  more 
difficult  for  people  to  see  that  7  was  the  only  other  choice  that 
could  invalidate  the  rule.  Many  people  turn  over  the  3  card,  and 
even  after  hearing  the  answer  and  having  it  explained,  some  subjects 
do  not  feel  right  about  it.  Johnson-Laird  and  wason  (1970)  think 
that  the  difficulty  stems  from  the  incomplete  "truth  tables"  which 
are  held  by  most  people,  and  from  their  set  for  truth  rather  than  for 
falsity.  If  the  arguments  are  presented  in  a  "practical  setting," 
say  with  real  objects,  then  the  logical  errors  decrease  sharply.  As 
far  as  we  know,  there  have  been  no  systematic  studies  of  the 
"negative  logic"  in  human  troubleshooting. 

This  review,  brief  as  it  is,  has  given  a  glimpse  of  behavioral 
research  that  is  relevant  to  troubleshooting.  Taken  together,  people 
who  work  on  troubleshooting  programs  and  devices  seem  never  to  refer 
to  it.  Humrro's  conclusion  from  over  twenty  years  ago  still  holds: 
"....the  study  of  human  problem  solving,  concept  formation, 
probability  learning,  and  so  forth,  has  provided  little  data  of 
iamediate  use  to  the  researcher  in  electronic  troubleshooting,  but 
has  provided  certain  limits  as  to  important  variables...."  (Czeh/ 
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1957) .  Rather  little  of  this  work  has  influenced  the  practical  design 
of  equipments,  courses,  and  troubleshooting  procedures.  Occasionally 
one  will  find  an  engineer  who  has  heard  of  the  half-split  notion,  and 
some  conception  of  a  learning  hierarchy  is  discernible  in  many  lesson 
plans.  But  such  items  as  "mnemonics  for  electronics,"  "story  coding" 
for  memory,  cognitive  sytle,  and  human  processing  of  conditionality 
and  falsification,  are  apparently  not  recognized  outside  of 
psychology.  This  could  be  due  simply  to  interdisciplinary  ignorance, 
but  then  it  is  very  difficult  to  take  a  conception  such  as  cognitive 
style,  or  "imagery-supported  electronics,"  and  to  develop  it  into 
something  useful  for  the  real  troubleshooting  domain.  Suppose  one 
wished  to  match  a  military  troubleshooting  aid  or  training  device  to 
an  individual's  cognitive  style.  There  would  have  to  be  a  lot  of 
psychometric  work,  difficult  field  trials,  and  long-term  commitment 
from  operational  carunands.  Instead  of  going  through  all  this,  the 
practical  operator  might  ask  his  technologists  to  set  up  the 
troubleshooting  task  so  as  to  reduce  the  impact  of  cognitive  style  on 
the  search  task.  That  is  exactly  what  the  aiders  have  done.  If  the 
ads  and  supports  are  effective,  then  individual  variance  in 
processing  skill  will  be  minimized,  or  at  least  reduced.  We  have 
already  noted  that  "concretizing"  the  P's  and  Q's  of  logic  problems 
will  tend  to  reduce  the  logical  errors,  to  improve  performance 
generally,  and  to  make  the  task  more  routine  for  everybody.  Hie 
practical  training  and  operational  executive  wants  this  kind  of 
aiding,  rather  than  an  academic  explication  of  the  factors  underlying 
the  performance  when  the  aiding  is  not  present. 
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Digital  systems  bring  special  problems  and  opportunities  to  the 
behavior  analyst.  A  large  digital  system  is  an  assembly  of  irany 
copies  of  a  few  standard  units.  There  are  only  about  a  dozen  basic 
logic  functions  such  as  AND,  OR,  NCR,  or  NAND.  Each  of  these 
functions  has  a  logical  truth  table,  and  can  be  realized  in  hardware 
by  means  of  various  technologies.  Usually  an  item  such  as  an  OR  gate 
is  encapsulated  as  a  small  integrated  circuit.  Operating 
sections  of  a  system  can  then  be  put  together  by  an  array  of  the 
simple  functions.  For  instance,  information  which  is  coded  in  four 
binary  digits  as  0's  and  l's  can  be  converted  to  decimal  form  by 
means  of  8  inverters  and  8  HAND  gates,  and  the  whole  converter  can  be 
smaller  than  a  letter,  or  even  a  dot,  on  this  page  (Coffron,  1979) . 

Understanding  a  digital  system  seems  to  be  a  psychological 
challenge  for  many  people.  It  takes  some  time  to  get  used  to  the 
many  logical  0's  and  l's  that  are  processed  and  transferred  about. 
Since  transistors,  resistors,  and  diodes  are  often  employed  in  the 
circuits,  the  student  also  must  know  enough  electronics  to  follow  the 
current  states  in  a  bistable  device  or  an  operational  amplifier. 
Often,  there  is  not  much  imagery  to  assist  human  memory  in  holding 
the  big  logic  arrays  in  mind.  The  digital  system  with  its  abstract 
design  may  seem  to  be  austere  and  counter-intuitive. 

Digital  components  such  as  counters  and  shift  registers  are 
driven  by  special  clocks,  and  so  the  timing  is  extremely  critical. 

The  clean  and  definite  nature  of  the  signals,  and  of  the  flip-flop 
circuits,  means  that  a  timing  diagram  of  signals  and  events  in  a 
circuit  can  be  very  precise.  A  timing  layout  may  show  free-running 
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clock  pulses  running  along  a  top  line,  with  real  time  going  to  the 
right.  Below  this  there  could  be  many  other  event  lines,  which  show 
preset,  clear,  and  output-synchronous  events.  Such  diagrams  seem  not 
to  be  very  memorable,  however,  and  the  meaning  of  all  the  time 
intervals  can  quickly  be  forgotten.  Furthermore,  unless  every  event 
is  understood  at  some  physical  action  level,  the  diagram  may  be 
useless  in  a  real  trouble  search,  and  the  technician  can  get  lost  in 
a  jumble  of  microsecond-duration  events.  The  cognitive  problem  in  digital 
electronics  seems  to  be  somewhat  similar  to  that  of  the  computer 
progranmer  (or  debugger)  who  must  at  once  understand  a  complex  program 
at  a  global  level,  yet  be  able  to  deal  with  the  most  minute  aspect  of 
one  or  more  individual  instructions. 

There  is  another  type  of  intellectual  effort  that  is  required  of 
any  electronic  technician,  and  particularly  so  of  one  working  on 
a  digital  system.  A  quote  from  an  engineering  text  will  give  the 
flavor : 

"....When  we  look  at  the  overall  system.... we  see  feed-back 
loops,  DAC's,  LED  displays,  and  digital  counters....  The 
troubleshooter  must  develop  the  ability  to  separate  in  his 
thinking  the  individual  subcircuits  from  the  complete 
circuit  or  system.  This  is  a  mental  separation  at  first  for 
the  purpose  of  understanding  the  small  circuit  and  large 
system  operation.  The  second  step  will  be  to  apply  the 
mental  separation  to  the  physical  hardware  to  accomplish  the 
task  of  troubleshooting."  (Coffron  1979). 

Again,  this  "mental  separation"  depends  on  a  detailed 
appreciation  of  circuit  functions  and  inter- relations,  at 
sane  level.  In  fact,  someone  who  could  troubleshoot  each  little  sub¬ 
circuit  would  probably  not  need  much  further  capability  to  do  the 
whole  thing.  There  is  a  big  picture,  of  course,  but  working  inside 
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the  big  picture  depends  on  many  little  perceptions  which  are 
accurate.  One  training  strategy,  which  is  often  used  in  the  computer 
industry,  is  to  teach  just  enough  electronic  theory  for  the 
technician  to  understand  the  dozen  basic  logic  funtions  in  digital 
processors,  or  perhaps  only  that  set  that  he  "has  to  know  about,"  in 
order  to  use  the  aids  furnished  to  him. 

There  are  special  problems  in  the  location  of  intermittent 
faults,  and  the  process  of  finding  them  is  just  beginning  to  be 
modeled  (Varshney,  1978) .  Many  such  faults  emerge  only  under  certain 
conditions  of  age,  timing,  and  adverse  operation.  The  necessity  for 
finding  intermittents  is  one  of  the  most  challenging  problems  the 
technician  faces  now,  and  this  will  probably  continue  into  the 
future.  From  theoretic*  onsi derations,  the  "memory"  require!  for 
intermittent  troubles  will  be  xuid  partly  in  the  technician's  head, 
and  partly  in  his  aids.  No  behavioral  research  has  been  done  in  this 
area. 

If  the  troubleshooter  of  the  imediate  future  will  indeed  be 
primarily  a  highly-skilled  user  of  complex  procedures  and  aidB,  then 
information  about  the  way  that  are  learned  should  be  useful 

in  preparing  such  people.  Again,  i  ere  are  pr anisine  ideas  and 
hints  from  academic  research,  but  not  much  that  is  easily  applied. 
Norman  (1978) ,  for  example,  says  that  the  oownon  sense  notion  of 
repeated  practice  as  the  key  to  learning  may  be  largely  erroneous. 
Instead  of  endless  repetition,  learning  often  proceeds  better  if  a 
■bory  line  is  napped  onto  the  new  material,  or  if  xnemonics  and 
visualization  can  be  utilized.  A  well-learned  complex  procedure  can 
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give  the  performer  a  play-like  experience  of  "flew,"  which  can  be 
intensely  satisfying  (Csikszentmihalyi,  1975).  Norman  believes  that  a 
student  should  know  more  about  his  own  psychology  of  learning,  and 
about  the  way  he  responds  to  practice  schedules,  feedback,  and  the 
arrangement  of  materials.  White  (1959)  points  out  that  being  able  to 
do  things  well  gives  one  a  feeling  of  "effectance,"  of  being  in 
control  of  one's  actions.  Since  White's  effectance  idea  requires 
that  the  performer  have  some  opportunity  for  originating  parts  of  a 
task  sequence,  the  designers  of  future  trouble-shooting  aids  might 
well  program  certain  possibilities  for  individual  choice  into  their 
conplex  test  routines.  The  best  troubleshooting  aid  might  be  one 
which  has  just  the  right  amount  of  cognitive  novelty  for  the  person 
using  it! 

It  is  well  known  that  there  are  ways  to  memorize  very  large 
quantities  of  data  by  utilizing  "peg-word  hooks"  as  in  the  Furst 
memory  system  (Logan,  1956) .  Even  if  a  large  table  of  symptom- 
function  relationships  were  memorized,  however,  the  problem  of 
correctly  exploiting  thoe  data  would  remain.  Considering  the  ease  of 
substituting  paper  or  microcircuitry  for  human  memory,  the  pursuit  of 
memory  enhancement ,  without  understanding,  seems  unattractive. 
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Engineering  h  Maintainable  Sysfcgn 
People  who  have  worked  on  human  factors  problems  in  complex 
equipment  often  have  disparate  views  of  the  world,  but  on  one  issue 
they  are  almost  universally  agreed:  almost  never  is  sufficient 
attention  given  to  "people  problems”  in  the  design  of  a  new  prime 
equipment.  A  little  reflection  and  observation  will  suggest  why  this 
is  so.  In  the  standard  R  &  D  scenario,  a  scientist  or  engineer  has 
a  bright  idea,  for  a  new  hardware  item.  After  some  company-funded 
preliminary  work  a  proposal  is  made  to  NASA,  FAA,  or  a  military 
service.  Contracts  are  let,  and  the  development  cycle  begins.  In 
the  early  stages,  nearly  the  total  emphasis  is  on  engineering 
feasibility,  and  such  matters  as  packaging,  logistics,  and  ease  of 
maintenance  always  "can  be  taken  care  of  later. r  Since  the  equipment 
is  complex  and  is  challenging  the  technical  limits,  there  will  be 
many  problems  in  getting  out  a  working  prototype,  and  by  then  there 
probably  will  be  time  and  cost  overruns.  If  the  hardware  conception 
is  basically  valid,  then  a  short  time  before  delivery  seme  attention 
will  be  given  to  support  functions,  and  the  maintainability 
specialist  and  other  support  people  will  be  allowed  to  look  at  the 
project.  By  then,  though,  radical  design  changes  may  be  expensive  or 
impossible,  and  quick  fixes  may  be  attempted.  Cnee  more,  an 
incomplete  system  goes  into  service. 

Of  course,  there  can  be  strong  persons  along  the  development 
trail  who  insist  on  maintenance  considerations  all  the  way.  At 
military  procurement  desks  these  are  often  people  who  have  had  to  use 
new  equipment  in  the  field,  and  who  are  sensitive  to  the  problems. 
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But  the  persons  with  the  technological  ideas  are  usually  engineers 
and  scientists  who  perceive  the  main  challenge  as  one  of  innovative 
design.  They  get  their  satisfactions,  and  their  reputations,  from 
the  design  activities.  The  program  managers,  on  the  other  hand,  have 
budget  and  delivery  schedules  in  mind,  as  well  as  acceptance  tests 
and  field  trials.  A  lot  of  money  and  prestige  is  riding  on  the  basic 
project  feasibility.  In  these  circumstances,  maintainability  is  just 
another  secondary  support  irritation  to  worry  about. 

There  is  a  "design  side"  to  maintainability  analysis,  and 
there  is  a  "system  availability"  side.  Availability  is  often 
expressed  as  a  joint  measure  of  reliability  and  maintainability. 

Thus,  if  reliability  is  denoted  by  Mean  Time  Between  Failure  (MTBF) 
and  if  maintainabil ity  is  measured  as  Mean  Time  To  Repair  (MWTR) , 
then  Availability  (A)  can  be  expressed  as  a  simple  ratio: 

MTBF _ 

A  «  WIBF  +  MTTR 

Formulas  like  this  can  be  computed,  graphed,  and  extended 
mathematically  in  many  ways.  As  just  one  example,  MTTR  is  often 
assumed  to  follow  a  log-normal  distribution,  and  under  this 
assumption  the  likelihoods  of  certain  availability  levels  may  be 
computed,  aquations  like  this  can  show  clearly  the  tradeoff  between 
reliability  and  maintainability,  and  it  is  no  doubt  interesting  to 
work  out  graphs  relating  availability  to  various  nunbers  of 
R  t  D  dollars.  But  such  formulas  are  rather  empty,  because  they  do 
not  tell  what  to  do  in  order  to  achieve  a  given  level  of  availability. 
Bence  we  do  not  pursue  the  availability  question  here,  except  to  say 
that  we  have  almost  never  seen  a  military  service  hold  a  manufacturer 


to  the  availability  promised  in  the  original  proposal. 

The  design  side  of  maintainability  engineering  is  a  much  richer 
and  more  useful  discipline.  It  encourages  the  formulation  of  an 
explicit  maintenance  policy,  it  gives  specific  design  rules  which  are 
(sometimes)  behaviorally  based,  and  it  can  stay  close  to  the  work 
that  must  be  performed.  When  fully  implemented,  a  maintainability 
effort  sta*  with  the  maintenance  concept  for  a  given  prime 
equipment.  The  ooncept  defines  levels  of  maintenance,  the 
arrangements  for  support  by  mobile  and  depot  facilities,  the  number 
of  operational  and  maintenance  installations,  spare  parts  facilities, 
and  other  system  requirements.  There  will  also  be  a  repair  policy. 

A  system  can  be  designed  to  be  nonmaintainable,  partially  repairable, 
or  fully  repairable  (a  nonmaintainable  item  is  not  repaired,  but 
simply  replaced  by  a  spare) .  A  partially  repairable  system  will  have 
some  assemblies  which  are  throwaway  items,  while  others  will  be  fixed 
when  they  fail.  Ideally  the  maintenance  ooncept  will  be  complete, 
that  is,  actual  behavior  constraints  required  at  the  different  levels 
wii:  be  fully  designated. 

.'  i' sequent  steps  in  a  good  maintainability  program  will  go  on  to 
tradeoffs,  design  assistance  and  review,  and  the  demonstration  of  a 
maintenance  capability  (Blanchard  &  Lowery,  1969) .  Check-lists  and 
hardware  evaluation  sheets  are  used  to  score  each  equipment  item,  in 
a  large  project,  there  will  be  much  paper  related  to  design 
reoQamndations,  and  regular  maintainability  meetings.  The  final 
demonstration  is  often  two-phased.  The  first  one  takes  plaoe  at  the 
contractor's  facility  and  uses  contractor  personnel;  the  second  is 
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held  in  the  user's  environment  such  as  a  Navy  ship,  with  the  user's 
people  in  charge. 

It  is  surprisingly  difficult  to  prove  the  utility  of  a 
maintainability  program.  There  are  many  case  studies  and 
illustrations  of  the  presumed  benefits  of  redesigning  an  equipment 
for  ease  of  maintenance.  A  favorite  textbook  example  is  an  Army 
field  radio,  which  held  many  arbitrarily  designed  and  located  switches 
and  dials  and  controls  on  its  front  panel.  To  operate  or  check  the 
original  radio,  a  technician  had  to  explore  the  entire  panel.  The 
task  sequence  was  made  more  regular,  allowing  the  operator  to  start 
at  the  top  and  work  clockwise  around  the  front  panel.  Readout  dials 
and  test  poir*  :  were  made  more  legible,  and  the  switches  were 
enlarged  arid  saaped  so  they  could  be  discriminated  by  a  gloved  hand 
in  the  dark.  Inside  the  box,  a  partial  rearrangement  of  wires  and 
parts  was  done,  so  that  the  electronic  states  were  more  clear-cut  to 
the  repairman,  and  use  of  clips  and  clamps  hastened  the  removal  of 
parts.  Yet,  nobody  ever  really  proved  these  improvements  resulted  in 
better  field  usefulness  of  that  radio. 

Maintainability  design  and  planning  effort  should  be  considered 
as  a  necessary  but  insufficient  condition  for  good  troubleshooting. 
The  cognitive  load  on  the  technician  is  the  key  thing.  It  does 
not  help  ti  technician  much  to  have  an  accessible  test  point  if  the 
Meaning  of  the  voltage  at  that  place  is  obscure,  or  if  he  does  not 
know  which  voltage  to  look  at.  Me  would  argue  for  a  basic  behavioral 
reference  for  many  of  the  design  proposals  made  by  maintainability 
P*>Ple. 
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Maintainability  programs  rarely  admit  a  fact  which  is  obvious  to 
others:  sometimes  the  corrective  maintenance  technician  has  an 
inpossible  task.  One  Navy  air-search  radar,  for  example,  could  not 
be  maintained  by  typical  technicians.  The  system  was  extremely 
complicated.  Designed  just  before  the  integrated-circuit 
technological  revolution,  it  had  several  cabinets  j armed  full  of 
subsystems.  Some  of  the  test  sets  accompanying  it  were  novel,  and 
difficult  to  use.  The  manuals  were  beautifully  bound  but 
impenetrable  to  the  average  technician.  Spares  were  often  very 
expensive,  which  made  a  throwaway  policy  untenable.  A  graduate 
electrical  enqineer  might  have  kept  the  radar  going  with  months  of 
specific  training  01.  it,  no  other  duties  to  perform,  and  a  big  parts 
budget.  Here  was  a  total  operational  failure,  yet  the 
maintainability  documents  and  field  advisories  kept  coming  in  a 
steady  flow,  both  from  the  manufacturer  and  the  Washington  bureaus. 
Nobody  ever  said,  out  loud,  that  the  system  was  not  maintainable. 

The  point  is  obvious:  real  troubleshooting  in  the  services  is  often 
done  by  people  who  have  a  year  or  less  in  a  military  technical  school 
and  some  months  in  the  field,  and  there  are  many  items  they  simply 
cannot  service.  When  the  maintainability  and  planning  documents  do 
deal  with  human  capabilities,  the  reference  is  often  to  formal  school 
levels  or  ratings,  with  assumptions  of  capability  in  the  higher 
enlisted  ranks.  The  assunptions  are  often  wrong;  many  first  class 
ET's  are  not  first  class  troubleshooters. 

Why  is  so  much  equipment  plainly  unsatisfactory  from  an 
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elementary  design  standpoint?  Do  the  engineers  not  know  of  our 
irainta inability  design  guides  and  review  procedures?  What  about  the 
design  suggestions  we  have  catalogued — are  they  seriously  inadequate 
or  inccnplete?  Maybe  the  best  answer  is  that  while  maintainability 
guides  and  directives  are  often  known  by  and  available  to  the 
designer,  there  is  not  a  sufficiently  clear  payoff  to  be  had  from 
consulting  them  in  a  systematic  way.  After  all,  much  of  the  advice  is 
rather  obvious:  Keep  it  simple,  label  things  clearly,  organize  check 
sequences  in  a  natural  order,  write  the  manuals  so  users  can 
understand  them,  encase  fragile  parts  in  plastic.  Glancing  through  a 
set  of  homilies  like  those,  a  professional  engineer  might  think  he 
was  not  getting  anything  novel.  Perhaps  an  even  more  basic  answer, 
and  a  major  key  to  eventual  utlization  of  the  advice  we  do  have,  is 
that  maintainability  people  are  staff  people,  and  seldom  have  the 
power  to  impose  their  arts  onto  a  design.  This  situation  can  change 
when  equipment  must  be  maintainable,  as  in  a  submarine  or  a  NASA 
space  laboratory.  Fortunately,  human  factors  courses  appear  more 
frequently  in  engineering  curricula,  and  are  also  popular  in  middle- 
management  military  courses.  Hie  most  critical  pressure  for 
application  will  come  from  the  users.  Unless  that  pressure  is 
applied,  good  maintainability  engineering  will  continue  to  be  very- 
rare. 


SECTION  III.  AIDING  AND  TRAINING  THE  TROUBLESHOOTER 
Aiding  Concepts  and  Devices 
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The  tech  manual  is  the  original  aid.  It  has  always  been  provided 
as  a  part  of  the  prime  equipment  package.  Much  effort  always  goes 
into  it,  and  yet  the  tech  manuals  have  usually  been  unsatisfactory  to 
the  user.  Maybe  this  is  because  there  are  many  users — naintenance 
technicians,  installers,  operators,  parts  suppliers,  field  engineers, 
field  change  people,  curriculum  planners,  and  so  on. 

While  the  information  in  a  manual  may  be  correct,  it  may  be 
inpossible  for  a  typical  user  to  find  out  just  what  he  wants  to 
know.  The  troubleshooting  section  of  the  manual  was,  and  often  still 
is,  not  arranged  well  for  a  military  technician  who  is  trying  to  find 
and  replace  a  faulty  unit  in  the  middle  of  the  night. 

Things  are  better  when  special  troubleshooting  manuals  are 
produced,  on  the  basis  of  behavioral  considerations  such  as  these 
(Rigney,  1970); 

1.  Content  organization  should  follow  the  sequence  of  events 
in  the  job.  The  first  things  in  the  book  should  be  the 
first  things  the  technician  does  as  the  search  proceeds. 
The  technician  works  through  the  book,  which  becomes 
more  detailed. 

2.  The  information  presented  should  be  relevant  to  the  job. 
Thus,  a  troubleshooter's  manual  should  include  only  the 
items  that  the  individual  needs. 

3.  The  level  of  discourse  should  be  appropriate  to  the  user's 
technical  knowledge.  A  trained  technician  probably  does 
not  know  much  electronics  theory,  and  he  cannot  use  an 
engineer's  course  on  theory  of  operation  of  the 
particular  system  he  is  working  on  now. 

4.  The  information  should  be  accurate.  An  idealised  wave¬ 
form  in  a  manual  may  not  be  a  good  basis  for  comparison 
with  a  real  wave  shape  at  a  test  point.  It  is 

often  surprisingly  difficult,  or  inpossible,  to  obtain  the 
good  ■normal"  parameters  as  shown  in  a  manual.  Failure  to 
obtain  the  reference  indications  can  cause  considerable 
confusion  for  the  technician. 
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5.  The  format  should  be  convenient  to  use,  in  the  user's 
environment.  Manuals  should  be  easy  to  carry,  to  read, 
and  to  prop  up  in  cramped  spaces.  Sometimes  a  "manual" 
could  be  something  like  a  pack  of  plastic  cards  riveted 
together  at  the  corners,  or  a  small  device  with  slides 
and  wheels. 

There  are  military  specifications  calling  for  such  improvements, 

of  course,  but  the  manuals  still  are  produced  by  much  the  same 

process,  and  the  same  people,  that  produced  the  older  inadequate 

ones.  DOD  has  devoted  large  sums  to  manual- improvement  projects  in 

recent  years,  such  as  "new  concept"  projects  in  manual  writing  and 

production  (Klesch,  1979) .  These  efforts  are  bf ginning  to  have  an 

inpact.  A  most  convincing  demonstration  by  Potter  (1976)  compared  a 

fully-proceduralized  (FPTA)  manual  and  a  logic-tree  manual  (LTTA) 

with  a  regular  Technical  Order  (TO) .  Both  new  formats  produced 

better  troubleshooting,  less  wasted  parts,  and  positive  acceptance  by 

the  field  people.  This  study  used  Air  Force  operational  radar  and 

computer  gear,  and  presented  realistic  fault-location  problems. 

About  the  only  disadvantages  of  the  new  formats  were  the  higher 

preparation  costs,  and  the  extreme  necessity  for  avoiding  errors  in 

the  documentation  (if  a  man  following  FPTA  instructions  is  told  to  go 

to  page  37  when  he  should  go  to  page  73,  he  may  be  irretrievably 

lost.)  Shriver' s  analysis  of  the  new  presentation  formats  indicates 

that  they  are  all  based  on  a  single  assumption: 

'....experts  can  analyze  the  equipment,  figure  out  what 
should  be  done  to  it  in  every  possible  situation,  and 
record  this  in  Technical  Manuals  that  even  a  novice  can 
use  correctly"  (Shriver,  1975). 

Shriver  goes  on  to  state  that  this  concept  is  realized  by  means 
of  four  techniques  for  producing  content  (task  analysis,  behavioral 
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cues  at  each  step,  etc.);  and  the  appreciation  of  this  content  is 
then  enhanced  by  employing  good  readability  and  display  considerations. 
Shriver  also  observes  that  the  technician  may  only  need  system 
understanding  at  a  block  level,  and  not  at  an  electron  level.  This 
accords  well  with  the  evidence  that  electron-level  knowledge  is  not 
well  retained  following  academic  training. 

Hand-held  Aids 

In  this  category,  we  think  of  items  like  the  flow  diagram  for  the 

t 

"B-7  fire  control"  (Morgan,  et  al.,  1963),  and  the  Experimental  Fault 
Locator  (XFL)  troubleshooting  wheel,  developed  by  Rigney  and  Fromer 

at  BTL  (Rigney,  et  al. ,  1965).  Morgan's  diagram  offered  a  clear 

t 

functional  layout  of  test  points  and  line  replaceable  units,  and  also 
furnished  a  sequence  of  checks  to  follow.  Rough  but  effective  test 
tolerances  were  also  provided.  For  example,  the  guide  says  that  at  a 

» 

testpoint  the  "wave  shape  is  not  critical;"  but  the  absence  of  a 
signal,  or  the  presence  of  spurious  oscillations,  does  indicate  a 
malfunction.  This  is  just  the  kind  of  information  that  a  technician 

* 

needs,  and  can  use. 

The  XFL  wheel  is  a  circular  plastic  device,  similar  to  a  circular 
slide  rule;  it  contains  a  symptom-malfunction  matrix  for  a  Navy 
transceiver,  along  with  an  algorithm  for  selecting  the  next  front 
panel  test  to  perform  until  the  fault  is  localized.  The  device  thus 
unburdens  the  technician  in  two  respects:  memory  for  the  malfunction- 
significant  symptoms,  and  the  formulation  of  a  logical  search  plan 
for  narrowing  down  the  fault.  Radio  operators,  who  had  no  training 
in  maintenance  work,  were  able  to  use  the  XFL  in  a  few  hours,  and 
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could  localize  faults  inserted  in  the  equipment.  This  finding  not 
only  indicated  the  value  of  the  aid,  but  also  confirmed  our  general 
thesis  that,  if  an  adequate  next-step  selection  aid  is  available  to  a 
technician,  good  troubleshooting  behavior  can  be  expected. 

Many  other  hand-held  devices  like  these  two  have  been  invented, 
and  some  have  reached  the  field.  The  original  research  studies 
almost  always  show  the  effectiveness  of  such  items,  but  somehow  the 
aids  themselves  rarely  become  widely  distributed  and  utilized.  One 
thing  is  certain:  One  cannot  sii^oly  develop  a  device,  send  it  to  a 
Navy  or  Army  field  unit,  and  assume  it  will  be  employed.  The  field 
people  have  to  be  prepared  for  the  device,  they  have  to  be  trained  in 
its  application,  and  most  important  of  all,  they  have  to  feel  a  need 
for  the  things  that  the  device  can  do. 

Perhaps  one  reason  that  aiding  devices  are  not  widely  seen  on  the 
job  is  that  they  have  been  somewhat  resistant  to  updating.  Ihe  aid 
may,  for  instance,  not  take  account  of  field  changes  in  the 
particular  prime  equipment.  If  the  technician  tries  to  use  it,  and 
encounters  an  error  or  omission,  he  may  (wisely)  discard  the  whole 
thing,  when  actually  it  would  work  most  of  the  time.  We  have  often 
heard  technicians  say  that  a  new  tech  manual  or  other  job  aid  is  "no 
good,"  but  subsequently  close  questioning  indicated  that  the  item  had 
never  been  given  a  fair  trial.  Sometimes  an  outside  observer  gets 
the  idea  that  technicians  in  the  field  are  too  suspicious  of  schemes 
and  devices  which  are  provided  to  them.  It  is  hard  to  say  how  much  of 
this  rejection  is  due  to  previous  experience  with  really  inadequate 
materials,  and  how  much  is  due  to  the  field  person's  need  to  maintain 
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his  own  discretion  and  control  over  his  work.  What  cannot 
be  ignored  is  that  something  about  the  field  environment  presents 
serious  difficulties  to  the  successful  introduction  of  a  job  aid. 

Right  now,  the  most  exciting  hand-held  aid  possibilities  are 
those  involving  micro- computers,  especially  those  termed 
programmable  calculators.  In  1979  technology,  a  small  memory  cube 
(read-only  memory  or  ROM)  about  a  half- inch  on  each  side,  holding 
nearly  a  thousand  program  instructions,  plugs  into  the  side  of  a 
small  hand-held  computer.  If  desired,  a  light  printer  can  be 
attached.  Hie  whole  thing  weighs  only  a  few  pounds,  is  ultra¬ 
reliable,  and  costs  less  than  400  dollars.  Special-purpose  ROM's 
approximately  the  size  of  sugar  cubes,  can  be  produced  economically 
in  quantities  over  a  few  hundred.  Thus,  we  envision  a  proliferation 
of  this  technology  which  transforms  a  general-purpose  processor  into 
a  special-purpose  machine. 

In  addition  to  commercial  applications  of  this  technology,  Towne 
(1977)  has  implemented  a  detailed  troubleshooting  strategy  for  the 
URC-32  tranceiver,  to  demonstrate  the  potential  of  guiding  the 
troubleshooter  with  a  hand-held  processor.  The  services  have  also 
begun  to  employ  this  technology  both  in  the  schools  and  the  fleet. 

The  latest  development  in  programmable  calculators,  the  Hewlett 
Packard  41 -C,  provides  non-volatile  memory,  eliminating  the  need  to 
load  a  program  each  day.  And  there  is  a  full  alphanumeric  display. 
This  latter  capability  allows  a  truly  interactive  program  in  which 
user  inputs  are  cued  by  natural  English  words  and  phrases.  While  not 
yet  built  into  a  programmable  calculator,  voioe  synthesizers  and 


generators  have  been  sufficiently  miniaturized  and  mass-produced  to 
be  incorporatd  in  the  next  generation  of  hand-held  devices. 
Human-Oriented  Computer  Aids 

The  term  "human-oriented"  is  used  here  to  differentiate  such 
systems  from  automatic  test  equipment,  which  is  usually  strictly 
hardware-oriented.  Until  a  few  years  ago,  "computer  assisted" 
usually  meant  that  an  aid  was  driven  by  its  own  prime  equipment. 

When  timeshare  technology  became  cheap  and  reliable,  it  became 
convenient  to  store  maintenance  charts  and  sequence  information  in  a 
master  program  which  could  advise  the  technician  about  the  best  next 
test  to  perform.  A  complete  system  like  this  becomes  a  more  or  less 
sophisticated  page  turner,  which  is  based  on  a  large  network  of 
contingent  action-choices.  Though  the  concept  seems  simple  enough, 
setting  up  a  practical  program  is  a  considerable  task.  A  good 
example  is  a  special  logic-tree  program  written  in  the  IBM 
Coursewriter  language  using  the  IBM  1440-1410  computer  and  a  1050 
student  terminal  for  storage  and  display  (Rigney  et  al.,  1966).  After 
only  an  hour  or  so  of  computer-aided  practice,  a  Chief  Radioman  who 
had  never  operated  the  URC-32  transceiver  was  able  to  work  through 
five  trouble  sequences  on  the  real  equipment,  which  was  standing  next 
to  the  computer  terminal.  All  except  one  of  these  problems  were 
"deep",  in  the  sense  that  they  required  correct  sequencing  and 
interpretation  of  at  least  ten  or  more  tests. 

This  demonstration  showed  that  special  languages  like  Course- 
writer  can  be  utilized  quickly  by  electronics  instructors  who  are  not 
trained  programmers,  and  that  even  a  few  hours  of  computer-guided 
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practice  can  show  interesting  behavioral  results.  For  each  trouble¬ 
shooting  problem,  however,  a  special  program  had  to  be  written  in 
order  to  provide  a  "trace"  through  the  symptom-malfunction  tree.  To 
generalize  this  guidance  function,  and  to  make  it  less  problem 
dependent,  Tcwne  and  King  of  BTL  produced  the  AIDE  configuration. 

AIDE  accepts  troubleshooting  tree  data  from  any  prime  equipment  in  a 
standardized  format,  and  furnishes  analyses  of  "next  best  check," 
detailed  task  specifications,  procedure  lists,  exercises  in  symptom 
interpretation,  exercises  in  equipment  set-up,  or  simulated 
maintenance  problems.  The  unique  goal  of  AIDE  was  to  serve  as  a 
performance  aid,  a  training  system,  or  some  mix  of  those  two 
functions,  as  determined  from  a  particular  user's  needs  and  experience. 
At  the  moment,  the  AIDE  demonstration  data  base  stores  125  color 
micrographic  images  and  thousands  of  words  of  appropriate  fault- 
locating  software.  This  is  enough  information  for  a  significant  part 
of  a  large  radar  system.  For  the  complete  radar  system  perhaps  1000 
images  would  be  needed,  but  there  would  be  no  additional  software  or 
conputer  memory. 

Another  advanced  computerized  add  is  the  LOGMDD  Diagnostic  Test 
Set,  which  was  designed  by  DePaul  and  his  associates,  as  a  means  of 
meeting  sixteen  desirable  attributes  of  a  performance  aid  (DePaul, 

1979) .  The  LOGMDD  hardware  is  about  the  same  size  as  a  15-inch 
portable  IV  set.  It  can  provide  test  sequences,  repair  parts 
listings,  reliability  data,  interpretation  of  test  readings,  and  so 
forth.  The  equivalent  of  500  pages  of  manual  information  can  be 
stored  on  one  5-3/2- inch  floppy  disk,  which  can  be  accessed  in 
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numerous  ways. 

lhere  is  surely  reason  for  optimism,  then,  on  the  technology  side 
of  the  aiding  question.  The  utilization  of  the  hand-held  computer, 
with  a  special  memory  cube  for  maintenance,  is  on  the  way.  And 
slightly  larger  systems  like  AIDE  and  DOGMDD  can  store  and  deliver  an 
immense  amount  of  information,  and  can  operate  in  several  aiding  and 
training  modes.  These  approaches  can  realize  all  the  behavioral 
efficiencies  promised  by  the  fully-proceduralized  manuals,  in  a  more 
responsive  format. 

Certain  problems  in  getting  all  this  to  the  user  will  persist, 
however.  Over  the  near  term,  success  or  failure  of  the  aiding 
technology  may  depend  heavily  on  the  military's  reaction  to  softer 
problems  of  implementation,  which  are  intrinsic  to  all  large 
organizations.  Suppose  a  new  aiding  idea  is  developed,  tested,  and 
ready  to  be  put  into  the  field.  Nobody  greets  it  with  open  arms. 

The  military  field  oomnander  has  long  since  adapted  to  marginal 
performance  as  the  standard,  and  he  knows  that  he  will  be  gone  before 
any  perceptible  benefits  will  come  from  this  new  idea.  There  are 
large  and  unwieldy  support  bureaus  who  might  be  threatened  by  the  new 
idea.  The  training  planners,  lesson-writers,  and  school  managers 
usually  will  not  be  eager  to  help;  they  may  fear  that  the  aid  will 
make  their  schools  apear  to  be  outmoded  or  unnecessary.  These  are 
conservative,  entrenched,  and  powerful  agencies;  the/  have  their  own 
departments  for  research  and  development,  which  it  is  in  their 
interest  to  preserve  and  expand.  They  will  say,  "Well,  we're  already 
doing  that."  They  will  think  the  idea  is  intruding  upon  their 
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dorain,  or  they  will  want  it  to  be  implemented ,  if  at  all,  by  a 
section  of  their  present  organization,  at  a  time  convenient  for 
them.  Or,  they  may  attempt  to  offer  a  "watered  down"  version  (Foley, 
1978) .  They  may  claim  that  introduction  of  the  new  idea  will  cause 
budget  difficulties,  or  they  may  prevent  access  to  research  subjects 
in  training  schools,  because  "....we  have  personnel  quotas  to 
deliver." 

Given  such  a  conservative  system,  what  can  be  done  by  the 
research  community?  One  thing  we  can  do  is  to  stay  closer  to  the 
real  operational  needs,  and  to  make  our  demonstrations  sharper,  so 
that  the  value  of  new  methods  is  clearly  cost-effective.  Training 
establishments  and  manual  writers  often  concentrate  on  delivering 
manpower  quotas  and  staying  within  budgets.  They  are  not  necessarily 
reinforced  for  solving  the  field  problems.  Inaba  (1979)  put  it 
plainly  enough: 

"Little  attention  is  given  to  the  usability  of  the 
manuals  or  the  performance  capabilities  of  the 
students.... If  a  change  is  made  and  the  technicians  become 
more  productive,  the  publications  and/or  training  manager 
do  not  share  in  the  benefits — except  possibly  for  a 
ccnpliment." 

But  this  means  that  somebody  who  does  concentrate  on  the  real 
operational  problems  may  find  that  he  can  make  an  impact  outside 
the  usual  bureaucracy.  A  strong  project  office,  for  example,  is 
often  able  to  implement  quickly  a  new  aiding  notion,  as  part  of  a 
hardware  procurement  package.  The  maintenance  concept  for  at  least 
■  one  Navy  shipboard  aonputer  system  was  implemented  in  this  way. 
Automatic  Test  Bauionent  (ATE) . 

Probably  no  single  maintenance  development  has  been  so  powerful, 
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and  yet  so  ineffective,  as  has  ATE.  A  sophisticated  ATE  routine  can 
sample  hundreds  of  measurements  in  a  short  time,  and  can  race  through 
a  whole  pre-launch  sequence  before  a  technician  could  get  a  meter  out 
of  its  case.  Often  the  ATE  system  really  can  locate  faults;  same 
programs  even  print  out  an  English  statement  of  just  what  needs  to  be 
done,  and  then  "thank  you"  for  doing  the  corrective  work.  Such 
narvelous  capabilities  lead  to  a  trap,  however,  by  suggesting  that 
since  ATE  can  do  so  much,  it  can  do  everything,  and  skilled  human 
troubleshooters  will  be  unnecessary.  This  supposition  is  false. 
Recent  history  proves  that  ATE  is  extremely  vulnerable  to  programming 
snarls,  prime  equipment  inaccessibility,  the  software  difficulties  of 
writing  fault  location  routines,  the  incompatibility  between 
automatic  and  manual  test  procedures,  and  so  forth. 

Many  good  analyses  of  ATE  development  issues  have  been  published 
(e.g.,  Myles,  1978;  Van  Hemel,  1978;  SETE,  1965).  The  overselling  of 
ATE  capabilties  still  goes  on,  however.  A  recent  example  is  the 
Versatile  Avionics  Shop  Test  (VAST)  concept.  This  is  an  expensive 
and  complex  general-purpose  test  system  for  avionics.  The  system  is 
an  engineering  tour  de  force  which  really  can  help  the  technician. 

But  consider  the  following  quote  about  the  training  of  people  who 
use  VAST  (Van  Hemel,  1979) : 

"The  original  concept  of  VAST  envisioned  the  use  of  an 
operator  with  minimal  training.... This  concept  has  been 
shown  lacking  because  it  anticipates  a  situation  in  which 
the  program  will  be  perfect.  The  machine  will  always 
operate  properly,  and  doc  mentation  associated  with  the 
testing  process  will  always  be  up-to-date  and  correct. 
Experience  has  shown  that  all  of  these  factors  seldom 
prevail  in  spite  of  the  most  stringent  efforts.... The 
assumptions  originally  made  concerning  the  VAST  main¬ 
tenance  technician  also  suffered  from  the  same  inac- 
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curacies  as  with  the  operator.... this  technician  required 
more  training  and  experience  to  effectively  troubleshoot 
the  complex  VAST  system." 

*  With  enough  money  and  effort,  systems  like  VAST  can  be  made  to 
operate.  But  they  seldom  achieve  their  expectations,  and  they  can  be 
just  another  source  of  frustration  to  the  field  people  because  they 

t  do  not  begin  from  a  behavioral-requirement  base.  Indeed,  an 

interesting  research  project  might  be  an  attempt  to  apply  human 
factors  and  technician-behavior  requirements  to  a  oomplex  ATE 

*  system,  during  the  design  phases.  As  far  as  we  know,  this  has  not 
yet  been  done  on  a  systematic  scale. 

t 
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Training 

Conventional  School  Training 

Readers  of  this  report  will  be  familiar  with  the  ordinary 
sequence  of  technical  schooling  in  the  Services,  and  with  the 
staggering  numbers  of  courses,  schools,  and  students.  The  DCD  is 
probably  the  biggest  schoolhouse  in  the  world  (Shriver,  1975) .  A 
recent  Defense  Sciences  Board  (Alluisi,  1976)  concluded  that 
technical  training  in  the  U.S.  military  was  usually  effective,  and 
one  can  accept  that  judgment.  From  a  research  standpoint,  there  are 
surprisingly  few  thorough  evaluations  of  the  training  system.  There 
are  indeed  many  body  counts,  and  schools  often  send  out 
questionnaires  to  the  using  commands,  asking  for  data  on  the 
proficiencies  and  deficiencies  of  school  graduates.  Such  information 
is  presumably  used  in  modifying  courses.  When  genuine  experimental 
evaluations  are  carried  out,  though,  they  are  often  done  in  a  context 
of  a  special  training  item,  such  as  an  aircraft  simulator.  Since 
realistic  simulators  are  very  expensive,  it  makes  sense  to  find  out 
whether  they  really  save  time  in  the  aircraft.  In  this  particular 
case,  acceptable  criterion  measures  can  be  defined,  and  the  utility, 
and  even  the  cost-effectiveness ,  of  a  training  device  can  be 
estimated  (Orlansky  and  String,  1976). 

But  with  no  criterion  to  guide  most  technical  training  courses, 
the  usual  curricular  approach  is  to  furnish  general  "theory" 
information,  with  the  expectation  that  this  information  will  be  the 
basis  for  on-the-job  specialization  later  (Shriver,  1975).  It  is 
hoped  the  technician  can  then  figure  out  what  to  do,  regardless  of 
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conditions.  Certainly  it  works  this  way  sometimes,  but  often  it  does 

*  not.  The  theory  which  is  learned,  and  most  likely  quickly  forgotten, 

may  be  so  indirectly  related  to  later  activities  as  to  be  irrelevant, 

and  there  may  be  no  direct  training  at  all  in  the  tasks  that  must  be 

9  performed.  The  present  training  system  is  beginning  to  respond  to 

such  circumstances,  although  rather  slowly.  Inaba  (1979)  notes,  for 

instance,  that  the  Navy  has  started  to  teach  the  manuals: 

9  "....the  course  essentially  leads  the  students 

through  the  manuals  (but)  the  emphasis  on  using  manuals 
in  training  has  not  really  helped  much  because  the 
manuals  are  not  usable.  Once  the  student  leaves  the 
training  environment,  the  manuals  are  put  back  on  the 
^  shelf  again. " 

The  two  main  scientific  difficulties  with  conventional  training 
courses  seem  to  be  (1)  the  lack  of  a  criterion  to  assess  the  adequacy 
of  the  teaching,  so  that  the  system  is  not  self-correcting;  and  (2) 
the  lack  of  a  clear-cut  behavioral  requirement  underlying  the 
training  plan.  Instead,  training  plans  begin  with  a  set  of 
assumptions  about  background  information  that  a  person  needs,  to  be 
able  to  learn  on  the  job. 

These  two  difficulties  conceivably  could  be  alleviated,  by 
research  or  by  fiat.  But  there  are  administrative  difficulties  as 

well.  To  repeat  an  earlier  point,  the  military  training  system 

\ 

is  essentially  a  monopoly,  with  its  own  power  and  autonomy.  It  does 
not  respond  quickly  to  a  new  method  oranew  requirement,  because  it 
does  not  have  to.  Also,  a  training  unit  may  not  easily  accept  inputs 
from,  say,  the  aiding  people  or  the  manual  writers,  in  defining 
course  content  or  practice  schedules.  The  trainer  may  think  that 
the  Be  other  people  want  to  eliminate  training  (Inaba,  1979) ,  and  so 
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he  may  go  ahead  with  his  "general  technical  information"  approach. 
After  all,  that  was  the  way  he  was  taught,  and  the  way  he  knows 
best.  Despite  such  conservatism,  we  should  be  slow  to  disparage 
conventional  training.  It  is  often  surprisingly  difficult,  for 
instance,  to  improve  upon  conventional  training  with  advanced 
educational  technology,  as  is  shown  in  the  next  section. 

CAI-CMI  Courses 

Computers  have  been  used  as  a  practical  school  aid  for  nearly 
fifteen  years.  The  first  successful  application  of  Computer  Aided 
Instruction  (CAI)  was  the  Suppes- Atkinson  project  at  Stanford,  which 
gave  arithmetic  and  spelling  lessons  to  young  children,  starting  in 
1965.  The  Palo  Alto  classroom  had  18  student  stations,  each  one 
equipped  with  its  own  terminal.  In  military  training,  there  have 
been  about  30  studies  of  CAI-CMI  (Computer  Managed  Instruction) 
effectiveness,  and  these  have  been  collected  and  exhaustively 
analyzed  by  Orlansky  and  String  (1979) .  Among  their  findings  are  the 
following: 

a.  Effectiveness  has  been  measured  only  by  testing  student 
achievement  at  school,  and  not  by  performance  on  the  job. 

b.  Student  achievement  in  CAI-CMI  courses  is  about  the  same 
as  that  observed  under  conventional  instruction. 

c.  CAI-CMI  courses  save  about  30  percent  (median  value)  of 
the  time  required  to  finish  the  same  course  given  under 
conventional  instruction;  the  time  savings  vary  across 
courses  and  situations. 

d.  Cost-effectivness  of  CAI -OH  has  not  been  conclusively 
demonstrated,  though  some  impressive  "avoided  cost” 
dollar  savings  can  be  claimed  for  the  Navy  CMI  system. 

These  results,  which  are  well  documented  in  the  Orlansky-String 
report,  should  temper  any  excessive  enthusiasm  for  CAI-CMI  courses  as 
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the  key  to  technical  training  efficiency.  Of  course,  the  oourse 
trials  that  have  been  reported  may  not  represent  the  most  advanced 
technology  available.  Truly  radical  CAl-Oll  improvements  may  await 
better  psychological  information  about  the  way  students  visualize  and 
absorb  complex  physical  and  mathematical  relations,  or  the  way  they 
learn  a  sequential  procedure.  Clever  scheduling  of  lessons  or  real¬ 
time  monitoring  of  performance  cannot  substitute  for  process 
information  about  the  learner. 

TVo  intriguing  minor  results  came  out  in  the  Orlansky-String 
survey  of  military  CAI-CMI.  One  of  these  was  that  CMI  was  about  as 
effective  as  CAI,  in  the  saving  of  student  training  time.  However. 
OH  assistance  costs  only  five  to  seven  cents  per  student  hour, 
whereas  CAI  costs  a  dollar  or  more  per  hour.  On  the  basis  of  this 
crude  comparison,  CMI  is  some  thirty  times  as  cost-effective  as 
CAI!  Of  oourse,  there  are  difficulties  with  such  a  simple-minded 
argument,  but  it  is  still  surprising  that  CAI  should  not  do  better 
than  CMI. 

Hie  other  finding  was  that  instructors  were  often  unfavorable  to 
both  CAI  and  OH.  Many  people  who  work  in  the  oonputer-aiding  field 
think  they  are  helping  the  instructors.  For  exanple,  the  claim  is 
often  made  that  by  taking  the  drill-and-practice  burden  off  the 
instructor's  shoulders,  the  computer  makes  his  job  easier,  and  allows 
him  to  concentrate  on  more  difficult  conceptual  teaching.  Why,  then, 
are  the  instructors  unfavorable,  and  are  their  negative  attitudes  due 
to  something  specific  about  the  oornputer-aiding  part  of  it,  to 
infamiliarity  with  the  system,  or  to  perceived  loss  of  oourse 
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control?  The  matter  deserves  careful  study.  Regardless  of  how 
corrputerized  a  course  is,  there  always  must  be  human  instructors 
around  to  load  and  run  it,  and  to  serve  as  consultants. 

Simulator  Trainers 

Maintenance  trainers  and  simulators  are  "poor  sisters"  in  the 
simulator  business.  Kinkade  (1979)  points  up  the  differences  between 
the  way  training  simulators  are  obtained  for  a  new  aircraft,  and  the 
way  they  come  about  for  maintainers  of  a  new  electronic  device.  In 
the  aircraft  case,  simulator  development  is  initiated  early  in 
procurement,  and  is  often  budgeted  and  evaluated  at  the  same  time  as  the 
aircraft.  The  DCD  aircraft  corrmunity  also  has  become  accustomed  to 
complex  and  expensive  aircraft  simulators,  with  the  average  cost 
running  over  a  million  dollars  (Orlansky  &  String,  1977).  Of  course, 
such  devices  are  often  worth  their  high  cost  because  of  the  great 
expense  and  trouble  of  operating  real  aircraft.  As  a  result  of  these 
circumstances,  the  NTEC  Directory  of  Naval  Training  Devices  shows 
over  70%  devoted  to  aircrew,  25%  supporting  other  operator  training, 
and  with  less  than  5%  dedicated  exclusively  to  maintenance 
technicians.  Furthermore,  of  those  devices  that  are  designated  as 
technician  trainers, 

"....most  are  two-dimensional  displays  of  system  schematics, 
or  'cutaway'  models  of  system  equipments.  Very  few  provide 
for  'hands-on'  practice  of  technician  tasks  and  these  almost 
exclusively  involve  simulation  of  actual  system  equipments." 

Studies  have  indicated  that  maintenance  trainer  simulators  can 

result  in  high  savings,  compared  to  using  the  prime  equipment 

(Daniels  &  Cronin,  1975?  Miller  &  Rockway,  1975) .  Also,  use  of  real 

prime  equipment  has  obvious  limitations.  The  hardware  will  require 
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"peaking  up"  between  training  sessions;  it  is  difficult  to  observe 
and  soore  performances;  the  training  model  which  is  available  may  not 
reflect  the  realities  of  field  installation,  and  so  forth.  Several 
projects  have  designed  complex  maintenance  simulator  trainers,  and 
conducted  seme  preliminary  evaluations.  Among  these  is  the  Fault 
Identification  Simulator  (FIS)  used  in  an  experimental  course  for 
automatic  boiler  control  technicians.  The  device  can  show,  in 
graphic  fashion,  the  symptoms  of  23  different  fault  conditions.  In 
fact,  all  possible  fault  conditions  resulting  from  a  single  component 
failure  in  the  actual  system  can  be  simulated  in  the  FIS  (Kinkade, 
1979) .  This  trainer  also  advises  the  students  of  incorrect  repair 
decisions  about  a  non-faulty  element.  Since  the  simulator  is  driven 
by  a  small  computer,  it  can  be  programmed  to  reflect  changes  in  the 
prime  equipment.  When  tried  out  as  part  of  an  experimental  course, 
the  class  finished  training  in  two  or  three  weeks,  as  compared  to  a 
regular  course  duration  of  five  weeks  (the  experimental  class  covered 
the  same  material  as  the  usual  course,  and  even  had  more 
troubleshooting  practice  than  in  the  old  system) . 

Shepherd  et  al.f  (1977)  simulated  the  control  panel  layout  of  a 
chemical  plant,  and  taught  trainees  how  to  recognize  faults  in 
three  different  ways;  (1)  a  theory  approach,  based  on  conventional 
instruction;  (2)  a  rules  model,  wherein  subjects  were  given  rules  that 
would  help  them  infer  failures  from  the  panel  array  and  (3)  a  "no 
story"  or  oontrol  group  who  received  no  theoretical  training.  The 
"rules"  groups  was  better  able  to  diagnose  unfamiliar  faults  (80% 
accuracy,  vs.  65%  and  49%  for  theory  and  oontrol) .  One  conclusion  is 


worth  quoting  here,  for  its  reiteration  of  a  previous  point: 

"....there  is  little  to  be  gained  by  teaching  'theory' 
alone.  Although  trainees  in  the  'theory'  group  initially 
performed  well  on  unfamiliar  faults,  their  score,  by  the 
final  test,  had  deteriorated  to  a  level  not  significantly 
different  from  the  'no  story'  group." 

The  AIDE  and  LOGMOD  systans,  and  other  advanced  aiding  devices 
with  large  memory,  also  can  be  used  as  simulator  trainers.  In  fact, 
pure  (exclusive)  training  and  pure  performance  aiding  are  rare 
occurrences.  Most  training  experiences  involve  some  degree  of  job 
support,  and  most  supported  performances  rely  on  some  degreee  of 
previous  training. 

Another  training  concept  is  the  Generalized  Maintenance  Trainer 
Simulator  (OITS) ,  (Rigney  et  al.,  1978).  This  system 
was  designed  to  give  students  intensive  troubleshooting  practice  in  a 
simulated  hands-on  training  environment.  Its  "simulational 
bandwidth"  is  confined  to  conditions  and  operations  necessary  for 
practicing  fault  localization,  either  from  front  panel  information  or 
from  data  at  test-points  behind  the  front  panel.  GMTS  adapts  readily 
to  simulate  different  kinds  of  equipment,  by  accepting  data  bases 
that  describe  functional  and  structural  architecture.  As  a  trainer, 
the  GMTS  functions  automatically.  Once  a  student  signs  on,  he  gets 
instructions,  his  progress  is  tracked,  and  a  detailed  performance 
record  is  kept.  In  a  trial  with  16  Navy  technicians,  practice  with 
the  GMTS  reduced  average  times  to  locate  UHF  malfunctions  from  20 
minutes  to  less  than  10  minutes.  Most  subjects  rated  it  favorably  as 
a  learning  aid  (Rigney,  et  al.,  1978). 

The  SCPHIE  training  system  is  one  of  the  most  "reactive"  of  the  present 
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simulator  conf igurations.  Subjects  can  change  component  values  in  the 
circuit  under  study,  can  propose  hypotheses  to  an  "articulate  expert", 
and  can  play  a  "troubleshooting  game"  by  predicting  the  effects  of 
inserted  faults.  Students  respond  very  positively  to  this  learning 
environment  (Brown,  et  al.,  1975).  The  simulated  tutor  usually  provides 
explanations  of  student  mistakes.  Some  tutors  even  proceed  in  a 
Socratic  fashion,  by  challenging  the  student  so  that  he  corrects  himself 
(Stevens  &  Collins,  1977) . 

These  several  recent  experiences  with  complex  maintenance  trainers 
are  most  encouraging.  Apparently,  special  maintenance  simulators  are 
able  to  teach  technicians  efficiently  about  malfunctions.  We  believe 
they  work  as  well  as  they  do  because  of  the  clear  explication  of 
synptom- fault  patterns,  and  because  of  the  intelligibility  appeal 
mentioned  by  Shriver  (1975).  Thus,  they  meet  Duncan's  suggestion 
that,  for  teaching  trouble  diagnosis,  training  simulators  should 
preserve  the  panel  layout  and  also  the  "logical  layout"  of  the 
indicators  (Duncan  et  al.,  1975b). 

On-the-Job  Training  (OJT) 

The  leading  research  on  OJT  is  probably  being  done  at  the  Rand 
Corporation,  where  Carpenter-Huffman  (1979)  and  others  have  been 
surveying  field  units,  instructors,  and  trainees.  Many  deficiencies 
in  present  OJT  practices  are  rather  to  be  expected:  The  training  is 
secondary  to  operational  needs  of  the  moment,  an  adequate  instructor 
may  not  be  present,  there  is  a  lack  of  supplies  for  training  purposes. 
There  is  also  concern  over  the  instructor-type  skills  of  the  trainers 
(Carpenter-Huffman,  1979); 


"The  most  common  single  complaint  about  QJT  on  the  TAC 
questionnaire  was  that  those  assigned  as  trainer  were 
inadequate  teachers." 

Underlying  these  rather  inevitable  deficiencies  are  seme 

administrative  causes.  Military  corrmanders, 

"....are  not  immediately  rewarded  for  the  quality  of  their 
maintenance  activities,  let  alone  their  maintenance  CUT 
programs.  Instead,  they  and  those  they  command  are 
penalized  if  documentation  of  progress  in  CUT  does  not 
conform  to  the  schedule  set  for  it. .. .pressures  to  come  in 
as  Categorty  1  on  the  Operational  Readiness  report  often 
destroy  the  validity  and  usefulness  of  vehicle  preventive 
maintenance  and  operational  readiries  records....  On  the 
other  hand,  since  progress  in  CJT  is  tied  to  the  promotion 
system,  supervisors  are  under  pressure  to  certify  a  person's 
competence  whether  or  not  the  certification  is  warranted." 

The  Rand  approach  to  CUT  involves  a  more  formal,  explicit,  and 

reinforcable  conception  of  CUT  within  operating  units.  Several 

successful  applications  appear  to  follow  this  approach,  including  the 

Navy  FRAMP  and  the  TOT  (Task-Oriented  Training)  projects  in  the  Air 

Force  Tactical  Air  Command.  Though  this  area  is  perhaps  not  as 

exciting  as  some  of  the  new  aiding  concepts,  it  is  receiving  some 

attention,  and  the  main  requirements  have  been  identified. 

Adninistrative  decisions  will  be  the  key  factors,  since  the  military 

can  have  effective  CUT  if  it  really  wants  it. 


SECTION  IV.  JOB  DESIGN  FOR  IHE  TROUBLESHOOTER 


Job  design  is  "....the  organization  of  a  job  to  satisfy  the 
technical-organization  requirements  of  the  work  to  be  accomplished 
and  the  human  requirements  of  the  person  performing  the  work"  (Davis 
i  Ranter,  1955).  The  second  part  of  this  definition  is  the  key  one. 


as  it  directs  attention  to  the  fit  between  the  work  and  the  needs  of 
the  worker.  Many  proposals  have  been  offered  for  improving  the  fit. 
For  example,  Davis  listed  twenty-three  techniques  which  might 
simultaneously  increase  worker  satisfaction  and  lead  to  more 
effective  output  (Davis  1957) .  Many  of  his  proposals  have  to  do  with 
increasing  worker  autonomy,  discretion,  achievement,  growth,  and  the 
meaningfulness  and  variety  of  tasks.  It  is  surprising  how  modern 
these  ideas  sound  after  more  than  twenty  years. 

Despite  the  absence  of  definitive  survey  data,  it  may  be 
worthwhile  to  speculate  about  the  present  job-design  status  of  the 
maintenance  technician's  work.  For  this  purpose,  we  consider  the 
three  job  characteristics  that  Hackman  and  Lawler  (1971)  identify  as 
contributing  to  worker  need  satisfaction  and  to  organizational 
goals.  Their  first  attribute  is  that  the  worker  must  feel  personally 
responsible  for  his  work.  On  this  dimension,  the  high-technology 
troubleshooter  should  score  fairly  high.  Take  the  computer 
maintenance  man.  He  often  works  alone,  and  he  may  not  turn  to  other 
people  for  help,  because  they  cannot  help.  At  the  same  time,  however, 
he  certainly  must  knew  that  he  is  primarily  a  skilled  user  of 
materials  which  were  originated  and  validated  by  others,  and  that  he 
is  not  ordinarily  free  to  vary  the  procedures  very  much.  Computer 
technicians,  for  example,  are  not  allowed  to  change  maintenance 
software,  though  we  have  heard  of  some  doing  so.  On  Gulowsen's  (1979) 
scale  of  work-group  autoneny,  these  technicians  would  come  out  near 
the  middle  of  the  range. 

The  second  Hackman- Lawler  job  requiranent  is  that  a  work  outcome 


J 


Page  66 


is  perceived  as  meaningful  and  worthwhile  to  the  individual . 

According  to  Turner  and  Lawrence  (1965) ,  meaningful  job  processes 
have  clear  beginning  and  ending  of  the  work,  they  utilize  skills  and 
abilities  which  the  worker  personally  values,  and  they  require 
considerable  variety.  On  these  grounds,  military  troubleshooters 
certainly  should  perceive  their  work  as  meaningful.  They  often  have 
camand  pressure  on  than,  and  this  accentuates  the  value  of  the  work 
being  performed. 

Feecback  is  a  third  job-design  criterion.  A  person  who  is  having 
higher-order  needs  satisfied  wants  to  know  how  he  or  she  is  doing. 
Superficially,  the  troubleshooter  gets  almost  perfect  objective 
feedback,  because  the  failed  machine  either  is  restored  or  it  is  not, 
and  the  technician  usually  knows  which  state  prevails  (but  not 
always,  as  we  indicated  earlier) .  This  iranediate  task  feedback,  of 
course,  is  not  necessarily  identical  with  the  performer's  own 
perception  of  his  effectiveness,  since  he  may  be  evaluating  himself 
against  some  other  standard. 

On  these  three  criteria,  then,  the  technician  should  be  rather 
highly  motivated  in  his  work,  and  from  superficial  indications  he  may 
indeed  be  quite  satisfied,  compared  to  other  military  people. 
Comparative  personnel  statistics  are  difficult  to  find,  but  one  large 
computer  company  reported  that  turnover  and  absence  rates  for 
commercial  maintenance  men  are  very  low,  lower  than  for  design 
engineers  or  programmers.  This  is  especially  so  for  medium-size 
cities  in  America.  A  few  on-site  interviews  with  roving  commercial 
computer  technicians  did  reveal  a  couple  of  cannon  sources  of 
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dissatisfaction.  One  of  these  was  the  technician's  non-professional 
status,  which  was  related  to  what  the  person  does.  Though  the  work  is 
technical  and  demanding,  experience  in  restoring  computers  results  in 
a  person's  learning  more  and  more  about  one  particular  company's 
conputers,  manuals,  and  fault-isolation  procedures.  But  it  does  not 
result  in  general  engineering  knowledge.  Thus,  many  technicians 
perceive  that  they  are  regarded  as  "less  than  engineers,”  and  that 
they  can  never  attain  engineer  status,  regardless  of  how  expert  they 
become  at  their  work.  The  military  services  may  be  missing  an 
opportunity  to  satisfy  these  growth  needs  of  their  technicians.  A 
similar  growth-need  situation  was  observed  in  the  U.S.  aerospace 
industry.  Engineers  in  that  industry  perceived  that  their  real  worth 
depended  in  their  state-of-the-art  design  capabilities.  As  a  result, 
when  design  work  was  to  be  done,  they  tended  to  design  all-new  items, 
and  to  reject  existing  and  off-the-shelf  hardware.  They  did  this  to 
improve  themselves  and  to  keep  up-to-date,  but  it  was  very  costly  to 
the  company.  A  solution  was  for  the  firm  to  consider  both  the 
engineer's  perceived  need  to  sharpen  intellectual  skills,  and  the 
demands  of  the  routine  design  work  which  had  to  be  done.  Management 
guaranteed  that  an  individual  would  have  career-advancing  assignments 
often,  though  at  any  one  time  he  might  be  doing  a  routine  job  (Davis 
&  Taylor,  1979).  A  similar  job  design  strategy  could  be  used  with 
military  maintenance  people. 

A  more  subtle  negative  factor  which  was  frequently  noted  in  the 
industrial  setting  was  the  psychological  distance  between  the 
maintenance  technicians,  and  other  people  in  the  computer  centers. 


Hie  roving  maintenance  person  may  have  few  close  relationships  with 
other  workers,  as  he  is  always  moving  from  one  place  to  another,  and 
the  other  technicians  back  at  maintenance  headquarters  are  also 
usually  on  the  move.  Some  respondents  even  confided  that  the  work 
leads  to  a  general  sense  of  isolation  and  non-sociability. 

As  a  final  job-design  variable,  we  mention  the  "flow"  experience, 

which  can  be  a  profound  dynamic  state  (Csikszentmihalyi,  1975) : 

"....the  holistic  sensation  that  people  feel  when  they 
act  with  total  involvement.... (the  person)  experiences 
it  as  a  unified  flowing  from  one  moment  to  the  next, 
in  which  he  is  in  oontrol  of  his  actions,  and  in  which 
there  is  little  distinction  between  self  and 
environment,  between  stimulus  and  response,  or  between 
past,  present,  and  future.... People  seek  flow 
primarily  for  itself,  not  for  the  incidental  extrinsic 
rewards  that  may  occur  from  it." 

Could  a  technician's  search  task  be  so  arranged  that  this 

autotelic  flow  state  could  occur  with  some  regularity?  If  so,  the 

motivational  advantages  should  be  enormous.  From  all  accounts,  flow 

activities  are  avidly  sought  by  the  people  who  have  experienced 

them.  A  beginning  could  be  made  by  analyzing  the  flow  potential  in 

certain  maintenance  activities. 

SECTION  V.  REPRISE  AND  RECOMMENDATIONS 

The  military  troubleshooter  is  a  fairly  good  searcher  and 
reasoner.  What  he  needs  most  is  a  clear  model  of  the  functional 
relations  among  symptoms  he  can  observe  and  malfunctions  in  the  prime 
equipment,  perhaps  accompanied  by  a  sequencing  model  for  ordering 
test  actions.  Often  it  is  impracticable  for  him  to  learn  or  remember 
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enough  theory  to  generate  this  model  for  himself,  and  so  the  model 

►  will  have  to  be  furnished  to  him.  The  best  procedure  for  doing  this 
so  far  is  to  have  the  prime  equipment  analyzed  in  such  a  way  that 
all,  or  nearly  all,  malfunctioning  states  of  the  equipment  can  be 

►  correlated  with  observable  symptoms.  The  states  and  symptoms  are 
then  stored  in  some  way  that  can  be  systematically  searched  via  a 
predetermined  sequence.  The  main  job  of  the  technician  then  becomes 

>  the  highly  skilled  task  of  searching  via  complex  instructions  and 
diagnostic  aids. 

"New  look"  formats  for  aiding  and  training  these  skills  have  been 

t  tried  out  in  the  laboratory  and  in  the  field,  and  they  are  clearly 

more  effective  than  conventional  training  plus  CUT.  The  critical 
requirement  is  the  accuracy  and  usefulness  of  the  information 

*  presented,  rather  than  the  particular  display  medium.  Present 
computer  technology  can  store  and  distribute,  in  either  hand-held  or 
larger  portable  units,  all  the  information  that  is  ordinarily 

I  included  in  book-size  manuals,  along  with  test  sequencing  and 

evaluating  data.  This  technology  can,  then,  alleviate  maintenance 
problems  on  most  complex  systems,  if  it  is  designed  and  applied  with 

*  the  behavioral  requirements  in  mind. 

The  military  should  not  be  satisfied  with  procedurally  aided 
troubleshooting,  regardless  of  how  effective  a  good  aiding  system 

I  may  be.  Many  maintenance  specialists  will  have  to  possess  real 

insight  into  the  complicated  devices  of  modem  warfare,  in  order  to 
attack  the  unforeseeable  problems.  The  provision  of  these  more 

*  advanced  people  will  be  a  continuing  challenge  to  the  research  and 

> 
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training  communities,  but  there  is  reason  for  optimism.  We  know  much 
more  about  the  interaction  of  men  and  machines  than  we  did  in  the 
vacuum-tube  era. 

Since  effective  technology  (psychological,  aiding  and 
maintainability)  is  indeed  available,  and  we  still  have  serious 
problems,  then,  there  must  be  administrative  or  other  reasons  why  the 
knowledge  has  not  been  more  fully  utilized  (Bond,  1970) .  We  believe 
that  it  is  not  cost  that  prevents  application.  Some  "new  look" 
aiding  and  training  materials  do  require  significant  preparation  and 
debugging  effort,  but  these  are  one-time  expenses,  and  they  are 
insignificant  compared  to  present  wasted  manpower  and  downtime 
losses.  The  administrative  key  is  this:  To  develop  and  to 
implement  effective  aiding  and  training  systems,  many  separate 
efforts  have  to  be  coordinated  and  directed  through  a  central 
"corrective  maintenance  control"  concept.  This  has  been  done  in  the 
cormercial  computer  world  and  in  other  domains,  but  it  cannot  be  done 
easily  in  the  military,  because  no  single  conmand  or  agency  is  really 
responsible  for  it  and  can  insist  that  a  valid  maintenance  concept  be 
realized.  The  procuring,  designing,  and  aiding  and  training  have  to 
be  planned  together  and  accomplished  together,  and  this  infrequently 
happens  in  the  present  military  system. 

In  a  report  addressed  to  the  research  oomunity,  most  of  our 
"reccrmendations"  come  in  the  form  of  projects  that  might  be 


undertaken.  A  few  of  these  have  to  do  with  basic  research  into  human 
appreciation  of  complex  devices,  others  attempt  to  further  the 
production  of  arrangements  which  will  facilitate  performance,  and 
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thus  bypass  the  psychological  problems.  No  doubt  some  of  these  ideas 
*  are  already  being  pursued.  The  projects  are  not  listed  in  order  of 

inportance.  For  immediate  practical  importance  to  military 
effectiveness,  numbers  4,  5,  10,  17,  and  19  would  be  near  the  top  of 
f  the  order.  As  far  as  psychological  significance  numbers  7,  12,  13, 

14,  and  16  are  most  interesting. 


t 
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Research  Recommendations:  Possible  Projects  in  Troubleshooting 

1.  Investigate  the  use  of  pseudo-verbal  software  routines  (e.g., 
Findler's  Puzzle-Solver,  Wang's  Theorem-Prover)  for  fault- location 
or  aiding,  t j  determine  if  the  pseudo-English  format  is  suitable 
for  practical  search  tasks. 

2.  Explicate  the  basic  relations  between  such  methodologically 
similar  concepts  such  as  Cohn-Ott  optimal  testing,  Fano  coding, 
Huffman  coding,  switching  theory  diagnosis  of  logical  networks, 
and  so  forth.  (Some  of  the  identities  among  these  methods  have 
already  been  proven,  but  nobody  has  put  than  all  together  for  the 
psychological  research  community.) 

3.  Explore  the  extent  to  which  small  handheld  aids,  such  as 
troubleshooting  sequence  cards  or  wheels,  are  now  applied  by  military 
technicians. 

4.  Find  out  how  to  best  utilize  hand-held  computer  technology  for 
troubleshooting.  Ibis  would  include  consideration  of  the  sizes 
of  procedural ized  logic  trees  that  could  be  stored,  the  best  ways 
to  utilize  display  and  recording  techniques,  acceptability  of  the 
device  to  field  people,  and  development  costs. 
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5.  Compare  the  effectiveness  of  "new  look"  procedural ized 
maintenance  manuals,  as  presented  via  a  LOGMDD  or  AIDE 
computerized  configuration,  with  the  same  information  given  in  a 
series  of  booklets. 

6.  Study  the  feasibility  of  a  strong  "Maintenance  Applications 
Group"  in  the  laboratories  for  each  service,  and  see  what  rewards 
could  be  given  for  high-quality  applications  research,  within 
civil-service  constraints. 

7.  Determine  suitable  methods  for  measuring  cognitive  style  in 
technical  work,  and  study  the  possibilities  for  exploiting  individual 
differences  on  this  variable,  in  the  maintenance  domain. 

8.  Review  a  suitable  sample  of  Automatic  Test  Equipment  (ATE) 
routines,  and  delineate  the  usefulness,  and  the  limitations,  of 
these  routines  to  ordinary  military  technicians. 

9.  According  to  Orlansky  and  String  (1979) ,  there  are  about  30 
studies  of  CAI  and  CMI  effectiveness.  Determine  how  closely  the 
CAI  and  CMI  technology  there  reviewed  is  "up  to  the  best  present 
state-of-the-art" . 

10.  Discover  the  reasons  for  unfavorable  ratings  of  CAI-CMI  programs 
by  the  instructors. 

11.  Study  the  process  by  which  good  ideas  are  "stopped"  in  the 
maintenance  world.  Suppose  somebody  proves  that  a  sequential  or 
logic-tree  troubleshooting  scheme  is  better  on  a  military 
radar  or  computer  (in  fact.  Potter's  group  did  this).  Determine 
why  the  technology  was  not  exploited  more  fully. 

12.  Determine  the  extent  to  which  electronics  or  electromechanical 
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theory  is  remembered,  after  schooling.  A  project  could  test 
at  several  intervals,  say,  3,6,  and  24  months  after  schooling 
is  finished. 

13.  Collect  all  the  means  for  helping  students  to  visualize  circuit 
and  electro-mechanical  relations,  and  evaluate  experimentally  the 
most  premising  of  these  methods.  There  are  many  ingenious 
proposals  from  mathematics  and  physics  teachers,  and  from  simmer 
science  workshops.  Seme  of  these  are  worth  experimental 
investigation,  and  could  be  correlated  to  the  literature  in 
cognitive  psychology. 

14.  Investigate  the  utility  of  "mnemonics  for  electronics,"  in 
memorizing  difficult  material  such  as  circuit  algebraic  laws. 

15.  Find  out  if  the  time  it  takes  a  technician  to  do  a  basic  digital- 
logic  operation  (via  Trabasso  or  Posner  timing  techniques)  is  a 
reliable  information-processing  indicant,  and  if  so,  whether  it 
is  correlated  with  troubleshooting  performance. 

16.  Explore  the  "flow"  and  "play"  aspects  of  troubleshooting 
performance,  according  to  the  Csikszentmihalyi  paradigm,  and  see 
if  aids  can  be  constructed  that  could  enhance  the  likelihood  of 
these  flew  effects  in  troubleshooting. 

17.  Do  an  extensive  comparison  of  military  troubleshooting  with  that 
performed  by  conmercial  computer  service  companies,  with  special 
attention  to  "what  the  military  can  learn"  from  the  commercial 
experience. 

18.  See  if  special  training  in  branch  matrix  reasoning,  can  transfer 
positively  to  logical  (troubleshooting)  search  in  realistic  S-M 
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tables. 

19.  Determine  the  effectiveness  curves  of  intensive  training  (more 
than  100  troubleshooting  problems,  perhaps)  in  using  a  complex 
diagnostic  aid  on  subsequent  troubleshooting  performance. 

20.  Originate  "story  line"  interpretations  of  complex  physical  events, 
say  in  AC  circuit  theory,  and  evaluate  these  for  their  enhancement 
of  learning. 


Administrative  Recommendations 

1.  Conduct  an  unbiased  survey  of  present  prime-equipment 
capabilities.  To  prevent  the  usual  "softening"  or  blurring  of 
lack  of  capability,  this  survey  probably  should  be  done  by  an 
outside  engineering  group.  Sampling  techniques  would  permit 
strong  inferences  about  true  capabilities. 

2.  Discover  and  ranove  non-maintainable  equipment  items.  This 
action  would  be  an  obvious  sequel  to  the  capability  survey.  It 
should  relieve  operating  commands  from  the  burden  of  carrying  non- 
usable  major  equipments. 

3.  Concentrate  first-enlistment  technician  activities  on  the  expert 
use  of  fully-proceduralized  troubleshooting  aids,  and  drastically 
reduce  or  eliminate  the  present  theory-oriented  courses.  Foley 
(1978)  shows  how  this  personnel  decision  alone  would  save  millions  of 
dollars  in  "effective  technician"  work  time.  Hard  data  supports  the 
idea;  in  addition  to  the  small  evaluation  studies,  the  services 
should  undertake  massive  trials  of  procedural ized  aids,  and  this 
probably  should  be  done  outside  the  present  training  oonmunity. 
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4.  Develop  a  "flying  squad"  troubleshooting  concept,  made  up  of 

>  truly  expert  technicians.  Unlike  the  first-enlistment  people, 

these  second-term  people  would  receive  extensive  theoretical 
training.  The  squad  would  handle  the  tough  field  problans,  that 

I  is,  the  remaining  troubles  that  cannot  be  readily  located  by 

procedural ized  aids.  We  do  not  refer  here  to  simple  seniority, 
such  as  a  cadre  of  first  class  ETs  with  conventional  training; 

►  we  have  many  of  those  now  who  cannot  solve  maintenance  problans. 

As  in  the  fully  proceduralized  concept  for  first- tenners,  the 
squad  concept  would  be  oriented  to  effective  performance,  in  much 

t  the  same  way  as  commercial  computer  trouble  squads  are  managed. 

The  prospects  of  qualifying  for  this  kind  of  work  should  be 
motivating  to  the  best  career  technicians,  and  could  lead  to 

I  professional-level  trainning  in  the  second  and  later  enlistments. 

5.  Plan  for  high-level  military  administrative  "fixers"  who  support 
a  maintenance  program,  and  continuously  guide  it  through  the 

♦  financial  and  administrative  difficulties  faced  by  every  change. 

A  fixer  might  be  an  admiral  or  general.  He  would  develop  a 
maintenance  constituency  for  supporting  the  use  of  special 

♦  troubleshooting  aids,  for  example.  He  would  also  intervene  with 

agency  officials  on  special  problems  that  arise,  such  as  finding 
money  to  debug  a  training  concept  or  hand-held  computer  aid,  or 
pursuing  implementation  plans  in  difficult  environments.  He 
would  be  able  to  move  across  departments  and  agencies,  and  would 
be  able  to  influence  and  control  activities  so  that  the  present 
technology  is  utilized. 

♦ 
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5001  Eisenhower  Blvd. 

Alexandria,  VA  22333 


1  Dr.  Robert  Sasmor 

U.  S.  Army  Research  Institute  for  th 
Behavioral  and  Social  Sciences 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joseph  Ward 

U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 


1  Dr .  Ed  Johnson 

Army  Research  Institute 
5001  Eisenhower  Blvd. 
Alexandria,  VA  22333 


1  Dr.  Michael  Kaplan 

U.S.  ARMY  RESEARCH  INSTITUTE 
5001  EISENHOWER  AVENUE 
ALEXANDRIA,  VA  22333 


1  Dr.  Milton  S.  Katz 

Training  Technical  Area 
U.S.  Army  Research  Institute 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333 
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Air  Force 


Air  Force 


i  1  U.S.  Air  Force  Office  of  Scientific  1  Brian  K.  Waters,  Lt  Col,  USAF 

Research  Air  War  College  (EDV) 

Life  Sciences  Directorate,  NL  Maxwell  AFB,  AL  36112 

Bolling  Air  Force  Base 
Washington,  DC  20332 

.  1  Dr.  Earl  A.  Alluisi 

HQ,  AFHRL  (AFSC) 

Brooks  AFB,  TX  7?? 35 

1  Dr  .  Phil  ip  De  Leo 
AFHRL/TT 

Lowry  AFB,  CO  80230 

1  Dr.  Genevieve  Haddad 
Program  Manager 
Life  Sciences  Directorate 
AFOSR 

f  Bolling  AFB,  DC  2033 2 

1  Dr.  Sylvia  R.  Mayer  ( TO 1 T ) 

HQ  Electronic  Systems  Div. 

Han  scorn  AFB 
Bedford,  MA  01731 

1  Dr.  Ross  L.  Morgan  (AFHRL/LR) 

Wright  -Patterson  AFB 
Ohio  45333 

1  Research  and  Measurment  Division 

t  Research  Branch,  AFMPC/MPCYPR 

Randolph  AFB,  TX  78148 

1  Dr.  Marty  Rockway  (AFHRL/TT) 

Lowry  AFB 
Colorado  80230 

1  Dr.  Frank  Schufletowski 

U.S.  Air  Force 
ATC/XPTD 

Randolph  AFB,  TX  78148 

,  2  3700  TCHTW/TTGH  Stop  32 

Sheppard  AFB,  TX  763H 

1  Jack  A.  Thorpe,  Maj.,  USAF 

Naval  War  College 
Providence,  RJ  02846 
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Marines 


1  H.  William  Greenup 

Education  Advisor  (E031) 
Education  Center,  MCDEC 
Quantico,  VA  22139 

1  Headquarters,  U.  S.  Marine  Corps 
Code  MPI-20 
Washington,  DC  20380 

1  Special  Assistant  for  Marine 
Corps  Matters 
Code  100M 

Office  of  Naval  Research 
800  N .  Quincy  St . 

Arlington,  VA  2221? 

1  DR.  A. L.  SLAFKOSKY 

SCIENTIFIC  ADVISOR  (CODE  RD-1) 
HO.  U.S.  MARINE  CORPS 
WASHINGTON,  DC  20380 


Other  DoD 


12  Defense  Documentation  Center 
Cameron  Station,  Bldg,  B 
Alexandria,  VA  22 319 
Attn:  TC 

1  Dr.  Craig  1.  Fields 

Advanced  Research  Projects  Agency 
1C00  Wilson  Blvd. 

Arlington,  VA  22209 

1  Dr.  Dexter  Fletcher 

ADVANCED  RESEARCH  PROJECTS  AGENCY 
1900  WILSON  BLVD. 

ARLINGTON,  VA  22209 

1  Director,  Research,  and  Data 
0ASD(MRA6L ) 

3B919.  The  Pentagon 
Washington,  DC  203C1 

1  Military  Assistant  for  Training  and 
Personnel  Technology 
Office  of  the  Under  Secretary  of  Defen 
for  Research  h  Engineering 
Room  3D129.  The  Pentagon 
Washington,  DC  PC 30 1 

1  HEAD,  SECTION  ON  MEDICAL  EDUCATION 
UNIFORMED  SERVICES  UN IV.  OF  THE 
HEALTH  SCIENCES 
6917  ARLINGTON  ROAD 
BETHESDA ,  MD  20019 


AD-AQB2  135  UNIVERSITY  OF  SOUTHERN  CALIFORNIA  REDONDO  BEACH  BEHAV— ETC  F/G  5/10 

TROUBLESHOOTING  COMPLEX  EQUIPMENT  IN  THE  MILITARY  SERVICES;  RES— ETC  (U) 
DEC  70  N  A  BONOr  D  M  TOVNE  N0001A-79-C-0237 

UNCLASSIFIED  TR-92  NL 
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Civil  Govt 


Non  Govt 


1  Dr.  Susan  Chipman 

Learning  and  Development 
National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  2020S 

1  Dr.  Joseph  I.  Lipson 
SEDR  W-638 

National  Science  Foundation 
Washington,  DC  20550 


1  Dr.  John  R.  Anderson 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Anderson,  Thomas  H. ,  Ph.D. 

Center  for  the  Study  of  Reading 
174  Children's  Research  Center 
51  Gerty  Drive 
Champiagn,  IL  61820 


1  Dr.  John  Mays 

National  Institute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1  Dr.  Arthur  Helmed 

National  Intitute  of  Education 
1200  19th  Street  NW 
Washington,  DC  20208 

1  Dr.  Andrew  R.  Molnar 
Science  Education  Dev. 
and  Research 

National  Science  Foundation 
Washington,  DC  20550 

1  Dr.  H.  Wallace  Sinaiko 
Program  Director 
Manpower  Research  and  Advisory 
Smithsonian  Institution 
801  North  Pitt  Street 
Alexandria.  VA  22314 

1  Dr.  Frank  Withrow 

U.  S.  Office  of  Education 
400  Maryland  Ave.  SW 
Washington,  DC  20202 

1  Dr.  Joseph  L.  Young,  Director 
Memory  A  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 


1  Dr.  John  Annett 

Department  of  Psychology 
University  of  Warwick 
Coventry  CV4  7AL 
ENGLAND 

1  DR.  MICHAEL  ATWOOD 

SCIENCE  APPLICATIONS  INSTITUTE 
40  DENVER  TECH.  CENTER  WEST 
7935  E.  PRENTICE  AVENUE 
ENGLEWOOD.  CO  80110 

1  1  psychological  research  unit 

Dept,  of  Defense  (Army  Office) 
Campbell  Park  Offices 
Canberra  ACT  2600,  Australia 

1  Dr.  Alan  Baddeley 

Medical  Research  Council 

Applied  Psychology  Unit 
15  Chaucer  Road 
Cambridge  CB2  2EF 
ENGLAND 

1  Dr.  Patricia  Baggett 

Department  of  Psychology 
University  of  Denver 
University  Park 
Denver,  CO  80208 

1  Mr  Avron  Barr 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  94305 
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Mon  Govt 


1  Dr.  John  Bergan 
School  of  Education 
University  of  Arizona 
Tuscon  AZ  85721 

1  Dr.  Lyle  Bourne 

Department  of  Psychology 
University  of  Colorado 
Boulder,  CO  80309 

1  Dr.  Kenneth  Bowles 

Institute  for  Information  Sciences 
University  of  California  at  San  Diego 
La  Jolla,  CA  92037 

1  DR.  JOHN  F.  BROCK 

Honeywell  Systems  4  Research  Center 
(MN  17-2318) 

2600  Ridgeway  Parkway 
Minneapolis,  MN  55“ 13 

1  Dr .  John  S.  Brown 

XEROX  Palo  Alto  Research  Center 
3333  Coyote  Road 
Palo  Alto,  CA  9“30“ 

1  Dr.  Anthony  Cancell i 
School  of  Education 
University  of  Arizona 
Tuscon,  AZ  85721 

1  Dr.  William  Chase 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Michel ine  Chi 
Learning  RAD  Center 
University  of  Pittsburgh 
3939  O'Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  William  Clancey 

Department  of  Computer  Science 
Stanford  (Jhiverslty 
Stanford,  CA  94305 


Non  Govt 


1  Dr.  Allan  M.  Collins 

Bolt  Beranek  4  Newman,  Inc. 

50  Houlton  Street 
Cambridge,  Ma  02138 

1  Dr.  Meredith  P.  Crawford 

American  Psychological  Association 
1200  17th  Street,  N.W. 

Washington,  DC  20036 

1  Dr.  Kenneth  B.  Cross 
Anacapa  Sciences,  Inc. 

P.C.  Drawer  0 

Santa  Barbara,  CA  93102 

1  Director 

Behavioural  Sciences  Division 
Defence  4  Civil  Institute  of 
Environmental  Medicine 
Post  orfice  Box  2000 
Downsview,  Ontario  M3M  3B9 
CANADA 

1  ERIC  Facility-Acquisitions 
“833  Rugby  Avenue 
Bethesda,  MD  2001“ 

1  Dr.  Ed  Feigenbaim 

Department  of  Computer  Science 
Stanford  University 
Stanford,  CA  9“305 

1  Mr.  Wallace  Feurzeig 

Bolt  Beranek  4  Newnan,  Inc. 

50  Moulton  St. 

Cambridge,  MA  02138 

1  Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville,  MD  20850 

1  Dr.  Edwin  A.  Fleishman 

Advanced  Research  Resources  Organ. 
Suite  900 

4330  East  Wiest  Highway 
Washington,  DC  20014 
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Non  Govt 


•  1  DR.  JOHN  D.  FOLLEY  JR. 

APPLIED  SCIENCES  ASSOCIATES  INC 
VALENCIA.  PA  16059 

1  Dr.  John  R.  Frederiksen 
Bolt  Beranek  A  Newnan 
D  50  Moulton  Street 

Cambridge.  MA  02138 

1  Dr.  Alinda  Friedman 

Department  of  Psychology 
University  of  Alberta 

•  Edmonton,  Alberta 
CANADA  T6G  2E9 


1  Dr.  Barbara  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90806 

1  Dr.  Frederick  Hayes-Roth 
The  Rand  Corporation 
1700  Main  Street 
Santa  Monica,  CA  90806 

1  Dr.  James  R.  Hoffman 

Department  of  Psychology 
University  of  Delaware 
Newark,  DE  19711 


1  DR.  ROBERT  GLASER 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
«  3939  O'HARA  STREET 

PITTSBURGH.  PA  15213 

1  Dr.  Marvin  D.  Glock 
217  Stone  Hall 
Cornell  Uilversity 
9  Ithaca,  NY  18853 

1  Dr.  Frank  E.  Gomer 

McDonnell  Douglas  Astronautics  Co. 

P.  0.  Box  516 
St.  Louis,  MO  63166 

i 

1  Dr.  Daniel  Gopher 

Industrial  A  Management  Engineering 
Teehnion-Israel  Institute  of  Technology 
Haifs 
ISRAEL 

(. 

1  DR.  JAMES  G.  GREENO 
LRDC 

UNIVERSITY  OF  PITTSBURGH 
3939  O'HARA  STREET 
PITTSBURGH.  PA  15213 
i: 

1  Dr.  Harold  Hawkins 

Department  of  Psychology 
university  of  Oregon 
Eugene  OR  97803 


1  Dr.  Earl  Hunt 

Dept,  of  Psychology 
University  of  Washington 
Seattle,  WA  93105 

1  DR.  KAY  INABA 
21116  VAN0WEN  ST 
CANOGA  PARK,  CA  91303 

3  Journal  Supplement  Abstract  Service 
American  Psychological  Association 
1200  17th  Street  N.U. 

Washington,  DC  20036 

1  Dr.  Wilson  A.  Judd 
McDonnell-Douglas 

Astronautics  Co. -St.  Louis 
P.0.  Box  30208 
Lowry  AFB,  CO  80230 

1  Dr.  Steven  W.  Keele 
Dept,  of  Psychology 
University  of  Oregon 
Eugene,  OR  97803 

1  Dr.  Walter  Kintsch 

Departaient  of  Psychology 
University  of  Colorado 
Boulder,  CO  80302 

1  Dr.  David  Kieras 

Department  of  Psychology 
University  of  Arlsona 
Tuscon ,  AZ  65721 
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Non  Govt 


1  Dr.  Mazie  Knerr 
Li tton-Mellonics 
Box  1286 

Springfield,  VA  22151 

1  Dr.  Stephen  Kosslyn 
Harvard  University 
Department  of  Psychology 
33  Kirkland  Street 
Cambridge,  MA  02138 

1  Mr.  Marlin  Kroger 
1117  Via  Goleta 

Palos  Verdes  Estates,  CA  9027** 

1  Dr.  Jill  Larkin 

Department  of  Psychology 
Carnegie  Mellon  University 
Pittsburgh,  PA  15213 

1  Dr.  Alan  Lesgold 
Learning  R&D  Center 
Uhlversity  of  Pittsburgh 
Pittsburgh,  PA  15260 

1  Dr.  Michael  Levine 

210  Education  Building 
University  of  Illinois 
Champaign,  IL  61820 

1  Dr.  Robert  A.  Levit 

Director,  Behavioral  Sciences 
The  BDM  Corporation 
7915  Jones  Branch  Drive 
Me Clean,  VA  22101 

1  Dr.  Charles  Lewis 

Faculteit  Sociale  Wetenschappen 
Rijksuniversiteit  Groningen 
Oude  Boter ingestraat 
Groningen 
NETHERLANDS 

1  Dr.  Mark  Miller 

Computer  Science  Laboratory 
Texas  Instruments,  Inc. 

Mail  Station  371.  P.0.  Box  225936 
Dalles,  TX  75265 


Non  Govt 


1  Dr.  Allen  Munro 

Behavioral  Technology  Laboratories 
18**5  Elena  Ave.,  Fourth  Floor 
Redondo  Beach,  CA  ‘90277 

1  Dr .  Donald  A  Norman 

Dept,  of  Psychology  C-009 
Univ.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Melvin  R.  Novick 

356  Lindquist  Center  for  Measurment 
University  of  Iowa 
Iowa  City,  IA  52 2*»2 

1  Dr.  Jesse  Orlansky 

Institute  for  Defense  Analyses 
400  Army  Navy  Drive 
Arlington.  VA  22202 

1  Dr.  Seymour  A.  Papert 

Massachusetts  Institute  of  Technology 
Artificial  Intelligence  Lab 
545  Technology  Square 
Cambridge,  MA  02139 

1  DR.  PETER  P0LS0N 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  COLORADO 
BOULDER,  CO  80309 

1  Dr .  Mark  D.  Reckase 

Educational  Psychology  Dept. 
University  of  Missouri-Colunbia 
4  Hill  Hall 
Columbia,  M0  65211 

1  Dr.  Fred  Re if 
SESAME 

c/o  Physics  Department 
Uhlversity  of  California 
Berkely,  CA  9**720 

1  Dr.  Andrew  M.  Rose 

American  Institutes  for  Resesrch 
1055  Thomas  Jefferson  St.  NW 
Washington,  DC  20007 
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Non  Govt 


1  Dr.  Ernst  Z.  Rothkopf 
Bell  Laboratories 
600  Mountain  Avenue 
Murray  Hill,  NJ  07974 

1  PROF.  FUMJKO  SAMEJIMA 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  TENNESSEE 
KNOXVILLE,  TN  37916 

1  DR.  WALTER  SCHNEIDER 
DEPT.  OF  PSYCHOLOGY 
UNIVERSITY  OF  ILLINOIS 
CHAMPAIGN,  IL  61820 

1  Dr.  Alan  Schoenfeld 

Department  of  Mathematics 
Hamilton  College 
Clinton,  NY  13323 

1  DR.  ROBERT  J.  SEIDEL 

INSTRUCTIONAL  TECHNOLOGY  GROUP 
HUMRRO 

300  N.  WASHINGTON  ST. 
ALEXANDRIA,  VA  22314 

1  Committee  on  Cognitive  Research 
%  Dr.  Lonnie  R.  Sherrod 
Social  Science  Research  Council 
605  Third  Avenue 
New  York,  NY  10016 

1  Dr.  Robert  Smith 

Department  of  Computer  Science 

Rutgers  Uiiversity 

New  Brunswick,  NJ  08903 
\ 

1  Dr.  Richard  Snow 
School  of  Education 
Stanford  University 
Stanford,  CA  94305 

1  Dr.  Kathryn  T.  Spoehr 
Department  of  Psychology 
Broun  University 
Providence,  R2  02912 


Non  Govt 


1  Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Yale  University 
Box  11A,  Yale  Station 
New  Haven,  CT  06520 

1  DR.  ALBERT  STEVENS 

BOLT  BERANEK  &  NEWMAN,  INC. 

50  MOULTON  STREET 
CAMBRIDGE,  MA  02138 

1  Dr.  David  Stone 
ED  236 

SONY,  Albany 
Albany,  NY  12222 

1  DR.  PATRICK  SUPPES 

INSTITUTE  FOR  MATHEMATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 

1  Dr.  Kikumi  Tatsuoka 

Computer  Based  Education  Research 
Laboratory 

252  Engineering  Research  Laboratory 
University  of  Illinois 
Urbana,  IL  61801 

1  Dr .  John  Thomas 

IBM  Thomas  J.  Watson  Research  Center 
P.0.  Box  218 

Yorktown  Heights,  NY  10593 

1  DR.  PERRY  THORNDYKE 
THE  RAND  CORPORATION 
1700  MAIN  STREET 
SANTA  MONICA,  CA  90406 

1  Dr.  J.  Uhlaner 

Perceptronics,  Inc. 

6271  Variel  Avenue 
Woodland  Hills,  CA  91364 

1  Dr.  Phyllis  Weaver 

Graduate  School  of  Education 
Harvard  University 
200  Larsen  Hall,  Appian  Way 
Cambridge,  HA  02138 
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1  Or.  David  J.  Veiss 
N660  Elliott  Hall 
University  of  Minnesota 
75  £.  River  Road 
Minneapolis,  MN  55455 

1  DR.  GERSHON  WELTMAN 
PERCEPTRONICS  IHC. 

6271  VARIEL  AVE. 

WOODLAND  HILLS,  CA  91367 

1  Dr.  Keith  T.  Wescourt 

Information  Sciences  Dept. 

The  Rand  Corporation 
1700  Main  St. 

Santa  Monica,  CA  90406 

1  Dr .  J.  Arthur  Woodward 
Department  of  Psychology 
University  of  California 

1  Counandlng  Officer 
ONR  Branch  Office 
ATTN:  Mr.  R.  Lawson 
1030  East  Green  Street 
Pasadena,  CA  91106 

1  Mr.  Phillip  Andrews 

Naval  Sea  Systems  Command 
NAVSEA  0341 

Washington,  D.C.  20362 

1  Mr.  Warren  Lewis 

Human  Engineering  Branch 
Code  8231 

Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

1  CDR  P.M.  Curran 
Code  604 

Human  Factors  Engineering  Division 
Naval  Air  Development  Center 
Warminster,  PA  18974 

1  Mr.  Ronald  A.  Erickson 
Human  Facotrs  Branch 
Code  3175 

Naval  Weapona  Canter 
China  Lake,  CA  93555 

1  Human  Factora  Department 
Coda  N215 

Naval  Training  Equipment  Canter 
Orlando,  FL  32813 


1  Commanding  Officer 
MCTSSA 

Marine  Corps  Base 
Camp  Pendleton,  CA  92055 

1  Dr.  Jerry  C.  Lamb 
Display  Branch 
Code  TD112 

Naval  Underqater  Systems  Center 
New  London,  CT  06320 

1  LT.  COL.  Joseph  A.  Birt 
Human  Engineering  Division 
USAF  AMRL 

Wright  Patterson  AFB,  Ohio  45433 


